Schizophrenia Research 143 (2013) 143-149

Contents lists available at SciVerse ScienceDirect

Schizophrenia Research

journal homepage: www.elsevier.com/locate/schres

On the HORYZON: Moderated online social therapy for long-term recovery in first episode psychosis

M. Alvarez-Jimenez '^1,l),:1 S. Bendall ^a,^b, R. Lederman ^c, G. Wadley^c, G. Chinnery ^d, S. Vargas ^d, M. Larkin ^a,^b E. Killackey ^a,b, P.D. McGorry ^a,b, J.F. Gleeson ^e

^a Centre for Youth Mental Health, The University of Melbourne, Australia

^b Orygen Youth Health Research Centre, Australia

^c The Department of Computing and Information Systems, The University of Melbourne, Australia

^d Orygen Youth Health, Australia

^e Australian Catholic University, School of Psychology, Melbourne, Australia

Background: Early intervention services have demonstrated improved outcomes in first episode psychosis (FEP); however, recent evidence shows that treatment benefits may not be sustainable over time. These findings have resulted in repeated recommendations for the implementation of longer term treatment programs. An Internet-based intervention specifically designed for young people with psychosis may provide a cost-effective alternative to prevent loss of treatment benefits from early intervention.

Methods: Our multi-disciplinary team has developed a highly novel online intervention (HORYZONS) in regular consultation with stakeholders within a specialist early psychosis program. HORYZONS integrates: i) peer-to-peer social networking, ii) individually tailored interactive psychosocial interventions, and iii) expert interdisciplinary and peer-moderation in a coherent platform designed to improve long-term outcomes in FEP. The acceptability, safety and initial clinical benefits of HORYZONS were examined through a 1-month pilot study with 20 participants with FEP.

Results: There were no dropouts during the pilot study. Seventy per cent of participants utilised the system for at least 3 weeks, 95% used the social networking features, and 60% completed at least 3 therapy modules. System usage was high during the study. There were no incidents and the majority of participants reported feeling safe, empowered and more socially connected using HORYZONS. Analysis revealed a significant reduction in depressive symptoms at follow-up.

Conclusions: Our results indicate that HORYZONS is feasible, engaging and safe and may augment social connectedness and empowerment in FEP. These findings have significant implications for the enhancement of specialist FEP services. The potential of HORYZONS to improve long-term recovery is worthy of further investigation.

© 2012 Elsevier B.V. All rights reserved.

1. Introduction

Over the preceding two decades, early intervention for psychosis has emerged as a major international psychiatric reform (McGorry and Yung, 2003). The transformation of psychiatric care has led to the proliferation of specialist first episode psychosis (FEP) services in Australia, Europe, North America and Asia (McGorry et al., 2008). Early psychosis services seek to provide timely entry into treatment and support young people in achieving both symptomatic remission and full functional recovery (McGorry et al., 1996). They adopt a youth-friendly and recovery-focussed model in order to empower young people to become more involved in their own treatment decisions and prevent the development of self-stigma, comorbid symptomatology and functional and social disability (McGorry et al., 2008). Controlled and quasi-experimental research has demonstrated that specialist FEP services improve psychotic symptoms (Petersen et al., 2005), reduce relapse rates (Craig et al., 2004; Petersen et al., 2005; Alvarez-Jimenez et al., 2011) and comorbid substance misuse (Petersen et al., 2005), and foster functional recovery (Melle et al., 2004), engagement with services (Craig et al., 2004) and patient satisfaction (Petersen et al., 2005), at least during the first 2 years of treatment.

However, the maintenance of treatment effects poses a major challenge to achieving the aims of early intervention (Singh, 2010; Gleeson et al., 2011). Patients with FEP are responsive to acute-phase treatments, but relapse rates are high (Alvarez-Jimenez et al., 2012a). Our group has developed effective cognitive-based interventions for relapse prevention in FEP (Gleeson et al., 2009), but treatment benefits were lost over time suggesting the interventions needed to be maintained over the longer term (Gleeson et al., 2011). More generally, specialist FEP services typically have treatment resources for only 18 months to 2 years, and recent reports suggest that the benefits of early intervention seen at the end of 2 years may not persist at 5 years after patients have been receiving generic follow-up (Bertelsen et al., 2008; Gafoor et al., 2010). These findings have triggered widespread calls for studies assessing the effects of longer term treatment programs (Bertelsen et al., 2008; Friis, 2010; Singh, 2010).

The positive effects of specialist FEP services are likely to persist when patients continue to receive specialised treatment (Linszen et al., 2001; Norman et al., 2011). This view is supported by the evidence that the first 5 years after psychosis onset constitutes a critical period determining longer term outcomes (Crumlish et al., 2009; Norman et al., 2011). Indeed, the period of risk for relapse extends well beyond the timeframe of specialist FEP services (Alvarez-Jimenez et al., 2012a). Furthermore, the termination of the specialised intervention and the transfer of care bring about feelings of loss for the patients (Bertelsen et al., 2008) and significantly derail engagement with treatment services (Singh, 2010), a pivotal element of early intervention programmes.

In this context, there is a clear need to fully realise the aims of early intervention by extending its effectiveness through innovative approaches (Kreyenbuhl et al., 2009; Singh, 2010). A stepped down level of care which bridges the gap between specialised intervention and standard treatment could provide a cost-effective alternative to prevent loss of treatment benefits and reduce the risk for disengagement from mental health services over the longer term (Norman and Malla, 2001). Internet-based interventions have the potential to realise this goal through providing cost-effective, non-stigmatising, constantly available support to young people suffering from psychosis.

This article aims to: i) provide a rationale for Internet-based interventions for psychosis and their potential to improve long-term recovery in FEP; ii) describe the development of a world-first online psychosocial intervention (HORYZONS) designed to maintain the benefits of early intervention beyond discharge from specialist FEP services; and iii) examine the feasibility, acceptability, safety and potential clinical benefits of HORYZONS in FEP patients through a pilot study.

2. Internet-based technologies and early intervention for psychosis

2.1. Are online interventions for first episode psychosis feasible?

The Internet and mobile technologies are developing at a phenomenal rate and hold great promise for influencing and even transforming treatment delivery in psychosis (Ben-Zeev, 2012). These technologies can be particularly useful for young people with FEP as individuals under 25 are by far the greatest users of Internet resources (Lloyd and Bill, 2004). Indeed, 95% of young Australians use the Internet daily (Ewing et al., 2008) and 96% of people aged 16-24 in the UK have used the Internet, with 86% using it every day or almost every day (Sharkey et al., 2011). Likewise, our data show that FEP patients show positive attitudes toward Internet-based interventions and, in particular, toward online social networking (Lederman et al., 2011).

2.2. What type of Internet-based intervention?

To date, despite their clinical and social potential, only a handful of studies have investigated the effects of evidence-based interventions for psychosis delivered through Internet-based technologies. These include online forums, computer-based psychoeducation, and mobilebased interventions.

2.2.1. Online peer-support groups

The development of peer support or user-led services is based on solid theory as well as promising research findings (Davidson et al., 2006). Peer support rests on the assumption that people who have overcome adversity can provide valuable support, guidance and hope to others facing similar difficulties (Davidson et al., 2006). Moreover, consistent with the helper-therapy principle (Riessman, 1965), being able to be of assistance to others can improve self-esteem and reduce experiences of self-stigma (Corrigan, 2006). Research on user-led services has found robust associations between peer support, empowerment, social networks and recovery in people suffering from psychosis (Corrigan, 2006). Social support has also shown to be a protective factor against psychotic relapse in FEP (Norman et al., 2005). While research on the effects of online peer-support groups for people with psychosis is sparse, preliminary evidence suggests that professionally moderated forums focussed on positive constructs such as self-efficacy, problem solving and social recovery are likely to produce the best outcomes (Alvarez-Jimenez et al., 2012b).

2.2.2. Online psychoeducation

Psychoeducation interventions for psychosis have been shown to improve medication compliance, reduce relapse rates, promote social functioning, and increase satisfaction with mental health services (Xia et al., 2011). Preliminary research further suggests that computerbased psychoeducation is acceptable, as effective as face-to-face or bibliotherapy, and preferred by some (Jones et al., 2001; Walker, 2006). Although more research is needed to ascertain their effective elements, online interventions which provide clear and engaging information and cater for level of insight are likely to produce better outcomes (Williams and Whitfield, 2001; Gonzalez-Blanch et al., 2007; Alvarez-Jimenez et al., 2009).

2.2.3. Mobile-based interventions

The characteristics of novel mobile devices including their accessibility, portability, affordability, online-connectivity, and ease of use provide an opportunity to widely deliver evidence-based interventions and enable real-time support in psychosis treatment (Alvarez-Jimenez et al., 2012b). Indeed, recent research shows that therapeutic interventions employing mobile devices are both acceptable to and efficiently used by people suffering from psychosis (Depp et al., 2010). Moreover, preliminary evidence shows that mobile phones and SMS-based interventions can be successfully used to monitor early warning signs (EWS) of relapse and prevent hospital admissions (Spaniel et al., 2008) as well as significantly improve medication adherence, socialisation and auditory hallucinations (Granholm et al., 2012).

2.2.4. Internet-based interventions and FEP

To date, no study has investigated the feasibility or effects of Internet-based interventions for FEP patients. Consequently, we conducted a focus group with young people with psychosis to examine their views on technology-based tools (Alvarez-Jimenez et al., 2012b). Qualitative analysis showed that FEP patients were enthusiastic about the use of Internet-based strategies as part of their treatment program. They clearly preferred a system characterised by social connectivity (i.e. users should be able to contact peers, share personal experiences, and contact clinicians if required). The system should resemble regular social networking programs (i.e. asynchronous, ongoing communication), but provide a private and enclosed social network. In addition, expert moderators should guide, but not censor, the interaction to ensure a safe and supportive environment. Finally, patients indicated that the system should provide useful and updated information relevant to their needs, and optional therapeutic interventions (i.e. cognitive-behavioural strategies). Conversely, concerns were raised about the potential risks of Internet-based strategies when experiencing active psychotic symptoms and about content which made them focus negatively on their condition (Alvarez-Jimenez et al., 2012b).

2.3. How to develop an Internet-based intervention for FEP patients?

A number of recommendations have been put forward to increase the likelihood of acceptance, safety and effectiveness of Internetbased interventions in psychosis (Valimaki et al., 2008). These include careful analysis of users' needs and preferences prior to development, active involvement of stakeholders throughout all phases of development (Valimaki et al., 2008), and regular consultations with potential users with regards to all aspects of the protocols (i.e. emergency responding, therapy/psychoeducation content and wording and specific features) (Depp et al., 2010). Careful attention should be paid to clinical safety and emergency procedures (Sharkey et al., 2011). Additionally, the relevance of multimodal induction procedures and pilot testing has been highlighted (Depp et al., 2010). Ultimately, Internet-based interventions for FEP that are specifically designed to supplement existing mental health services and augment traditional relationships through online interaction are likely to be most beneficial (Alvarez-Jimenez et al., 2012b).

3. Development of the MOST model and the HORYZONS system

In line with the aforementioned recommendations, we have developed a new conceptual model of on-line behavioural interventions entitled “Moderated On-line Social Therapy" (MOST). The MOST model integrates: i) peer-to-peer on-line social networking; ii) individually tailored interactive psychosocial interventions; and (iii) involvement of expert mental health and peer moderators to ensure the safety of the intervention. The elements of the MOST model have been applied to a world-first on-line system entitled HORYZONS, designed with the purpose of maintaining the clinical benefits of early intervention.

HORYZONS has been developed by a multidisciplinary team of clinical psychologists, computer programmers, health informatics experts, web and graphic designers, and professional writers. In accordance with international recommendations, the online system has been developed iteratively over a 30-month period following participatory design principles (Schuler and Namioka, 1993) through a process of regular feedback and testing with focus groups of stakeholders (i.e. FEP patients and clinicians). HORYZONS includes fully functional versions for computer and Internet-enabled mobile devices (i.e. smart-phones and tablets).

3.1. Characteristics of HORYZONS

The HORYZONS system adopts the MOST conceptual model of online interventions. Thus, HORYZONS comprises a platform for the delivery of a range of evidence-based and interactive psychosocial interventions which are enhanced by a moderated on-line social networking environment. Specifically, HORYZONS integrates: i) peer-to-peer on-line social networking; ii) individually tailored interactive psychosocial interventions; and (iii) expert moderation.

3.1.1. Interactive psychosocial interventions

On entry into the system the patient answers a set of standardised questions that guide the delivery of tailored interactive psychoeducation modules. Modules are informed by evidence-based psychosocial interventions developed by our group (Killackey et al., 2008; Gleeson et al., 2009) and target key risk factors for psychotic relapse and salient domains in the early recovery process including: (a) psychoeducation, (b) early warning signs of relapse (EWS), (c) depression, (d) social anxiety and (e) stress management (Table 1). Some interactive modules are mandatory (i.e. psycho-education or ‘How minds work') while others are optional (e.g. ‘Early Warning Signs' (EWS) of Relapse) and their delivery is guided by user preferences. In addition, the content of modules is tailored to individual clinical characteristics. For example, specifically designed “psychoeducation" or “EWS" modules have been developed for participants with high insight vs. low insight to foster their engagement with the system (Alvarez-Jimenez et al., 2009). Finally, the therapeutic objectives of the modules are bolstered through discussion with peers in the ‘cafe’ (online environment) under the guidance of ‘coaches' (expert moderators).

3.1.2. Peer-to-peer on-line social networking (the ‘cafe’)

The ‘cafe’ includes a web feed (or newsfeed) where clients and moderators can post comments, information, upload pictures and videos, and ‘like’ different content. The newsfeed incorporates categories that organise discussion threads into relevant themes (e.g. ‘what's on your mind’, ‘I'm loving right now’, ‘strengths news’, ‘group news’) (eFig. 1). Moreover the system includes a ‘wall’ function displaying the activity of individual users, a ‘homepage’ which shows all the activity and notifications relevant to the user, a ‘network’ (similar to a ‘friends’ function) and a ‘job zone’ where users are provided with information regarding training and vocational recovery and access online to a vocational rehabilitation expert (eFig. 2). HORYZONS further incorporates a number of moderated social networking features that are progressively unlocked as patients advance through the therapy modules. For example, participants are invited to join moderated online problem solving groups. Nominated issues are discussed through structured phases following an evidence-based problem solving framework (McFarlane et al., 1995; McFarlane, 2002). Offered solutions and users' experiences are stored in a data-base serving as a ‘wiki’ for participants. Moreover, participants are encouraged to share personal coping resources for stressful events (i.e. ‘what works for me’) which are accumulated and categorised for future reference.

Importantly, HORYZONS adopts a strength-based approach (Seligman et al., 2006) through which users are guided and prompted to identify, discuss and exercise key personal strengths within the online environment and in real-life to enhance self-esteem, foster social functioning (Hall and Tarrier, 2003) and reduce depression (Seligman et al., 2006) (eFig. 3). This approach is based on the novel positive psychology model which poses that psychosocial interventions should aim to build strengths, meaning and purpose as well as relieve symptoms (Lee Duckworth etal., 2005; Seligman et al., 2006). Preliminary research suggests that this model is particularly appropriate for online interventions. For example, FEP patients may not engage effectively in online content which make them focus negatively on their condition (Alvarez-Jimenez et al., 2012b). Further, excessive expressions of fear and anxiety online increase stress and reduce quality of life (Lieberman and Goldstein, 2006), while Internet interventions focussed on self-efficacy have produced improved clinical outcomes in patients with psychosis (Rotondi et al., 2010).

3.1.3. Expert moderation

HORYZONS uniquely provides a ‘hub’ in which clinical psychologists and vocational workers collaboratively moderate the system with the purpose of improving both clinical and vocational functioning. Their role is to provide guidance, monitor participant's clinical status and ensure the safety of the social networking environment. HORYZONS moderation follows a theory-driven model known as ‘supportive accountability’ which poses that human support enhances engagement through accountability to a moderator who is perceived as trustworthy, benevolent and having expertise. Accountability involves clear, process-focussed and user driven expectations that take into account patient motivation (i.e. level of support is inversely proportional to the patient's intrinsic motivation) (Mohr et al., 2011). HORYZONS is moderated on a daily basis (i.e. 2 h/day during week-days, and 1 h/day during weekends).

3.1.4. HORYZONS safety protocol

The HORYZONS safety protocol follows best practice in Internet research involving vulnerable people (Sharkey et al., 2011) and is composed of 3 levels of security including system security, online safety, and clinical safety. System security conforms to industry best practice as defined by the Open Web Application Security Project (OWASP). Online safety is managed in accordance with the guidelines put forward by the Australian Communications and Media Authority (ACMA).

Clinical safety is ensured through manual and automated procedures. First, information related to clinical risk is screened daily by moderators. Second, HORYZONS incorporates an automatic alert system which detects information consistent with increased risk of relapse or suicide (via regular monitoring of EWS of relapse and self-harm-related terms using previously validated approaches (Goh and Huang, 2009; Huang et al., 2007). Any detected increased risk activates the HORYZONS crisis protocol that includes an initial risk assessment and, where necessary, liaison with suitable emergency services. Details on the safety protocol and ethical issues have been provided elsewhere (Gleeson et al., in press).

4. The HORYZONS pilot study

The purpose of the pilot study was to conduct an initial evaluation of HORYZONS regarding its feasibility, acceptability, safety and potential clinical utility for FEP patients. We hypothesised that HORYZONS would be: 1) feasible and favourably received; 2) regularly used; 3) safe; and 4) viewed as a useful tool for long-term recovery and social connectedness.

4.1. Participants

The sample included 20 patients recruited from the Early Psychosis Prevention and Intervention Centre (EPPIC, Melbourne). Inclusion criteria for the study were: (a) a first episode of a DSM-IV (APA, 1994) psychotic disorder or mood disorder with psychotic features; (b) aged 15-25 years inclusive; (c) <6 months treatment with an antipsychotic medication prior to registration with the early psychosis service; and (d) remission of positive symptoms of psychosis, defined as scores of 3 (mild) or below on the subscale items hallucinations, unusual thought disorder, conceptual disorganisation, and suspiciousness on the Brief Psychiatric Rating Scale (BPRS) (Lukoff et al., 1986) over that last 4 weeks. Additional inclusion criteria to ensure low level of risk within HORYZONS included: (e) low aggressiveness, defined by a score of 4 or below on the hostility subscale of the expanded version of the BPRS for the month prior to study entry; and (f) low suicidal risk defined as a score of 4 or below on the BPRS suicidality subscale for the month preceding study entry. Exclusion criteria included: (a) intellectual disability and (b) inability to converse in, or read English. Additional exclusion criteria to ensure safety within the online system included (c) a DSM-IV (APA, 1994) diagnosis of either antisocial personality disorder (ASPD) or (d) borderline personality disorder (BPD).

The mean age was 20.3 years (SD = 2.7) with 50% male. Selfreported ethnicity was 45% Anglo Australian, 25% Asian, 15% European, 5% African and 10% Bi-racial. Thirty five per cent of the samples were unemployed with 100% never married. Ninety per cent of the samples were living with their family and 10% living with friends. The study was approved by the Melbourne Health Human Research Ethics Committee. Participants provided informed consent while continuing treatment within the EPPIC programme.

4.2. Design and procedures

This study used an uncontrolled single-group design to evaluate feasibility, acceptability, safety and initial benefits of a pilot psychosocial intervention (Mueser and Drake, 2005). Study participants were recruited over a 3-week period. Eligible participants were invited to participate by the study research assistant (RA) after the online application was briefly showcased. Once informed consent was obtained, the RA provided participants with logon details, helped them set up their account and oriented them to the HORYZONS system, including details of the terms of use. Online moderators welcomed new users and encouraged existing users to interact with them within 24 h of enrolment.

HORYZONS was monitored daily (i.e. 2 h/day during week-days, and 1 h/day during weekends) by clinical psychologists and a vocational worker. Moderation integrity was ensured through a detailed moderation manual and weekly group supervision sessions with senior clinical researchers (MA-J and JF-G) from the research team. Participants were assessed at baseline and at 4 weeks follow-up on outcomes described below.

4.3. Measures

Baseline DSM-IV diagnosis was assessed via the Structured Clinical Interview for DSM-IV (SCID patient version) (First et al., 1996). Feasibility and acceptance of HORYZONS were tracked measuring participants' usage of the online system (i.e. frequency, duration and patterns of use). In addition, a questionnaire and semi-structured interview were developed based on the User Experience (UX) approach (Bargas-Avila and Hornbsk, 2011) which assessed the following themes: 1) helpfulness; 2) easy-of-use; 3) attractiveness; 4) safety; 5) social interaction; and 6) further suggestions. Each domain was assessed through a number of statements which the respondent rated on a Likert scale from strongly agree to strongly disagree as well as open-ended questions. Good acceptability of the system was defined a priori as at least 10 of the 20 participants logging on weekly during the 1-month pilot.

Safety was assessed through the following indicators: 1) adverse events (i.e. psychotic relapse and clinical deterioration related to usage of the system), 2) inappropriate use, and 3) participants' reports on inappropriate content.

Symptom rating measures included the 24-item version of the Brief Psychiatric Rating Scale (BPRS) (Lukoff et al., 1986), the clinician-rated Calgary Depression Scale for Schizophrenia (CDSS) (Addington et al., 1993), and the Beck Anxiety Inventory (BAI) (Beck et al., 1988).

4.4. Analysis

Frequency, duration and patterns of use were tracked in real time. Up-to-date charts were visible on demand to researchers and moderators via the website. Quantitative and qualitative feedbacks from the questionnaire were analysed and reviewed for themes related to usability, perceived benefits, safety, and challenges of online interventions. Paired samples t-tests were conducted and within-group effect sizes reported for statistically significant changes between baseline and post-test in clinical measures.

4.5. Results

4.5.1. Feasibility and acceptance

Data on the usage of HORYZONS showed that 60% of participants utilised the system over 4 weeks and 70% for at least 3 out of 4 weeks, with a total of 275 logins during the trial (Table 2). Participants provided positive ratings for the ease of use, perceived utility, and enjoyment of HORYZONS. The majority (75%) reported that they had a positive and constructive experience using HORYZONS, 90% would recommend it to others, and no participants reported negative or challenging experiences.

The social networking features were used by 95% of the sample, with a total of 371 postings and 4170 social page views or actions (i.e. posts, ‘like’) during the study. There was a median of 192 (range = 0-702) social pages views/actions per participant (Table 2). Accordingly, 70% perceived social networking as useful, and 90% considered moderation to be supportive. Interestingly, 85% considered that it would be beneficial to include moderators who were previous users of HORYZONS and 90% reported they would like to become online peer-moderators.

With respect to therapy module usage, 95% completed at least 1 full therapy module, 60% completed 3 at least modules, 45% completed 4 modules, and 15% completed all 7 modules (Table 1). There were a total of 1390 therapy-based page views, with a median of 65 (range = 7-169) per participant (Table 2).

4.5.2. Safety

No incidents (i.e. adverse events or inappropriate usage) were reported during the study. Similarly, analysis showed no worsening of positive psychotic symptoms at 1-month follow-up (Table 3). One participant experienced a psychotic relapse; however, both the participant and the treating team considered the clinical worsening unrelated to the use of HORYZONS. This is further supported by the low usage of HORYZONS by the relapsing participant, with only 2 logins during the trial. Finally, 100% participants considered HORYZONS to be both safe and confidential, and 90% reported that moderation contributed to the safety of the system.

4.5.3. Potential clinical utility

Analysis of symptom rating measures revealed a moderate to large improvement in participants' depressive symptoms at 1-month follow-up (Table 2). Moreover, 60% reported that HORYZONS significantly increased their social connectedness, 55% felt empowered in their own recovery process, and 70% considered the system to be a useful long-term treatment option beyond discharge.

5. Discussion

To the best of our knowledge, this is the first study to develop and test an online intervention specifically designed for FEP patients, and is possibly a world-first intervention using the MOST model. HORYZONS has been developed by a multidisciplinary team of experts in constant consultation with stakeholders. It uniquely integrates peer-to-peer online social networking, evidence-based interventions and professional and peer support in a coherent, innovative platform designed to provide ongoing support and prevent disengagement from mental health services. The results of this pilot study demonstrated HORYZONS to be highly acceptable and appealing to young FEP patients. More than two thirds of the sample regularly used the online system and all participants reported feeling safe and supported in doing so. This was further reflected in the highly successful rate of recruitment to the study and the volume of traffic on the site.

Data on system usage and therapy completion strongly supported the feasibility of HORYZONS. There were no drop-outs during the pilot trial and completion of therapy modules was relatively high, with 60% of participants completing at least 3 full therapy modules within one month. This compares well with the standard weekly dosage of psychotherapy provided in individual therapy studies for FEP (Jackson et al., 2005, 2007; Edwards et al., 2006; Gleeson et al., 2009; Penn et al., 2010), particularly considering that HORYZONS

Table 2

Logins and individual usage of the main components of HORYZONS (n = 20) over the 1-month pilot study.^a

^a Covers period from February 1,2012, to 16th of March 16,2012; participants were recruited into the system from February 1, 2012, to February 17, 2012.

^b Percentage of participants with more than 6 logins.

^c Number of page reads (page ‘hits’).

^d Percentage of participants with more than 30 page hits or posts.

Table 3

Means (M), standard deviation (S.D.), and within-group effect sizes (Cohen's d) for clinical measures (n = 20).

BPRS = BriefPsychiatric Rating Scale; CDRS = Calgary Depression Rating Scale; BAI = Beck Anxiety Inventory.

participants undertake therapy modules independently and without the active prompts of therapeutic sessions.

Our attrition and module completion rates compare very favourably to unsupported, open-access, short-term Internet-based programs (Eysenbach, 2005; Christensen et al., 2006), and even online interventions incorporating support and tracking by interviewers or counselors (Christensen et al., 2004; Clarke et al., 2005). Likewise, usage of HORYZONS (i.e. frequency of use in relation to time) compared well to a previous web-based intervention for people with schizophrenia which included online educational materials and a therapy forum (Rotondi et al., 2010). While the short duration of the trial and the characteristics of the clinical sample may partially account for these positive findings, these results suggest that the novel integration of online moderation, peer-to-peer social networking and therapy modules (i.e. the MOST model) provides a promising and engaging platform for delivering online interventions in FEP.

Analysis of clinical variables showed a moderate to large reduction in depressive symptoms at 1-month follow-up (Table 2). Subsequent analysis of individual-level data revealed that 7 of 10 participants with reduced depression scores were regular users of HORYZONS. Whereas the uncontrolled design of the study does not allow causal attributions, the clinical significance of the latter finding should be noted. Depression is pervasive in the early course of psychosis (Upthegrove et al., 2010) and individual interventions for FEP have shown only modest effects in reducing depressive symptoms (Jackson et al., 2005; Edwards et al., 2006; Gleeson et al., 2009, 2011). Taken together, this suggests that possibly the increased social connectedness (as reported by 60% of participants) or the empowering and strengths-based model adopted by HORYZONS (55% of participants reported feeling empowered in their own recovery using the online system) may have contributed to the reduction in depressive symptoms at follow-up. The potential of HORYZONS to improve depressive symptoms in FEP is worthy of further investigation.

The current study has several limitations. First, as noted above, the uncontrolled design precluded any causal inferences about the efficacy of HORYZONS (i.e. reduction in depression may be a function of either time or background interventions), however this was never the intended purpose of the study. Second, the short-term duration of the study precluded examination of the long-term effects and retention of the intervention. Thirdly, the current findings regarding acceptability, safety and potential clinical benefits can only be generalised to remitted FEP patients with low risk of aggression and suicidality. Fourth, the semi-structured interview was administered by an assessor non-blind to the study hypothesis, which may have over-estimated positive findings, and participant responses were subject to demand effects, social desirability and recall bias. That said, considerable efforts were made to minimise these biases. Namely, i) prior to interviews it was emphasised that honest responses were most beneficial for future users and versions of HORYZONS; ii) interviews were conducted showing HORYZONS through an Internet-enabled tablet to minimise recall bias; iii) all interviews were recorded, transcribed and rated by an independent rater; and iv) system usage, qualitative data and responses to the questionnaire were matched for consistency. Notwithstanding these caveats, our pilot study provided “proof of concept” for HORYZONS in terms of its acceptability and potential clinical benefits and warrants a full-scale trial in which these limitations can be addressed.

5.1. Conclusions and future research

The results of this study showed HORYZONS to be a promising online intervention for FEP patients as it yielded high acceptance and usage, low attrition, high satisfaction, and perceived increased social connectedness and empowerment. Participants reported feeling safe and perceived HORYZONS as a useful treatment strategy beyond discharge from a specialised FEP service.

In the near future we will evaluate whether HORYZONS is an effective strategy for the maintenance of specialised treatment effects in FEP. In the meantime the results of our pilot study highlight the potential of HORYZONS to: i) significantly improve the clinical and cost-effectiveness of specialised FEP services; ii) improve access and ongoing engagement of FEP patients in evidence-based preventative interventions; and iii) and transform current models of treatment maintenance in FEP.

Role of funding Source

This study was supported by generous funding from the Telstra Foundation, the Helen MacPherson Smith Trust, the University of Melbourne, the Telematics Trust, The Institute for a Broadband-Enabled Society (IBES), and the Colonial Foundation. The sponsors did not participate in the design or conduct of this study; in the collection, management, analysis, or interpretation of data; in the writing of the manuscript; or in the preparation, review, approval, or decision to submit this manuscript for publication.

Contributors

MA-J and JF-G supervised the study and wrote the first draft of the manuscript. S-B, R-L, G-W, G-C, S-V and M-L significantly contributed to the development and evaluation of HORYZONS. E-K and PD-M contributed to the design of the study and critically revised the manuscript. All authors contributed to and have approved the final manuscript.

Conflict of interest

The authors report no additional financial or other affiliation relevant to the subject of this article.

Supplementary data to this article can be found online at http://dx.doi.org/10. 1016/j.schres.2012.10.009.

Acknowledgements

The authors wish to thank Christopher Miles, Kathryn Junor and Sylvia Collinety for their contribution to the development of HORYZONS, and the study participants for their valuable contributions to this research.

References

ACMA, Australian Communications Media Authority, y. www.acma.gov.au.

Addington, D., Addington, J., Matickatyndale, E., 1993. Assessing depression in schizophrenia — the Calgary Depression Scale. Br. J. Psychiatry 163, 39-44.

Alvarez-Jimenez, M., Gleeson, J.F., Cotton, S., Wade, D., Gee, D., Pearce, T., Crisp, K., Spiliotacopoulos, D., Newman, B., McGorry, P.D., 2009. Predictors of adherence to cognitive-behavioural therapy in first-episode psychosis. Can. J. Psychiatry 54 (10), 710-718.

Alvarez-Jimenez, M., Parker, A.G., Hetrick, S.E., McGorry, P.D., Gleeson, J.F., 2011. Preventing the second episode: a systematic review and meta-analysis of psychosocial and pharmacological trials in first-episode psychosis. Schizophr. Bull. 37 (3), 619-630.

Alvarez-Jimenez, M., Priede, A., Hetrick, S.E., Bendall, S., Killackey, E., Parker, A.G., McGorry, P.D., Gleeson, J.F., 2012a. Risk factors for relapse following treatment for first episode psychosis: a systematic review and meta-analysis of longitudinal studies. Schizophr. Res. (as).

Alvarez-Jimenez, M., Gleeson, J.F., Bendall, S., Lederman, R., Wadley, G., Killackey, E., McGorry, P.D., 2012b. Internet-based interventions for psychosis: a sneak-peek into the future. Psychiatr. Clin. North Am. 35 (3), 735-747.

APA, 1994. Diagnostic and Statistical Manual of Mental Disorders, 4 ed. American Psychiatric Association, Washington, DC.

Bargas-Avila, J.A., Hornbsk, K.P., 2011. Old Wine in New Bottles or Novel Challenges? A Critical Analysis of Empirical Studies of User Experience. ACM CHI Conference, Vancouver.

Beck, A.T., Epstein, N., Brown, G., Steer, R.A., 1988. An inventory for measuring clinical anxiety: psychometric properties. J. Consult. Clin. Psychol. 56 (6), 893-897.

Ben-Zeev, D., 2012. Mobile technologies in the study, assessment, and treatment of schizophrenia. Schizophr. Bull. 38 (3), 384-385.

Bertelsen, M., Jeppesen, P., Petersen, L., Thorup, A., Ohlenschlaeger, J., le Quach, P., Christensen, T.O., Krarup, G., Jorgensen, P., Nordentoft, M., 2008. Five-year follow-up of a randomized multicenter trial of intensive early intervention vs standard treatment for patients with a first episode of psychotic illness: the OPUS trial. Arch. Gen. Psychiatry 65 (7), 762-771.

Birchwood, M., Smith, J., Macmillan, F., Hogg, B., Prasad, R., Harvey, C., Bering, S., 1989. Predicting relapse in schizophrenia: the development and implementation of an early signs monitoring system using patients and families as observers, a preliminary investigation. Psychol. Med. 19 (3), 649-656.

Christensen, H., Griffiths, K.M., Korten, A.E., Brittliffe, K., Groves, C., 2004. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J. Med. Internet Res. 6 (4), e46.

Christensen, H., Griffiths, K.M., Mackinnon, A.J., Brittliffe, K., 2006. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol. Med. 36 (12), 1737-1746.

Clarke, G., Eubanks, D., Reid, E., Kelleher, C., O'Connor, E., DeBar, L.L., Lynch, F., Nunley, S., Gullion, C., 2005. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J. Med. Internet Res. 7 (2), e16.

Corrigan, P.W., 2006. Impact of consumer-operated services on empowerment and recovery of people with psychiatric disabilities. Psychiatr. Serv. 57 (10), 1493-1496.

Craig, T.K., Garety, P., Power, P., Rahaman, N., Colbert, S., Fornells Ambrojo, M., Dunn, G., 2004. The Lambeth Early Onset (LEO) Team: randomised controlled trial of the effectiveness of specialised care for early psychosis. BMJ 329 (7474), 1067.

Crumlish, N., Whitty, P., Clarke, M., Browne, S., Kamali, M., Gervin, M., McTigue, O., Kinsella, A., Waddington, J.L., Larkin, C., O'Callaghan, E., 2009. Beyond the critical period: longitudinal study of 8-year outcome in first-episode non-affective psychosis. Br. J. Psychiatry 194 (1), 18-24.

Csikszentmihalyi, M., 1990. Finding Flow: the Psychology of Engagement with Everyday Life. Harper and Row, New York.

Davidson, L., Chinman, M., Sells, D., Rowe, M., 2006. Peer support among adults with serious mental illness: a report from the field. Schizophr. Bull. 32 (3), 443-450.

Depp, C.A., Mausbach, B., Granholm, E., Cardenas, V., Ben-Zeev, D., Patterson, T.L., Lebowitz, B.D., Jeste, D.V., 2010. Mobile interventions for severe mental illness: design and preliminary data from three approaches. J. Nerv. Ment. Dis. 198 (10), 715-721.

Edwards, J., Elkins, K., M, H., Harrigan, S.M., Donovan, K., Athanasopoulos, O., McGorry, P.D., 2006. Randomised controlled trial of a cannabis-focused intervention for young people with first-episode psychosis. Acta Psychiatr. Scand. 114, 109-117.

Ewing, S., Thomas, J., Schiessl, J., 2008. CCI Digital Futures Report: the Internet in Australia. ARC Centre of Excellence for Creative Industries and Innovation Institute for Social Research. Swinburne University of Technology, Melbourne.

Eysenbach, G., 2005. The law of attrition. J. Med. Internet Res. 7 (1), e11.

First, M.B., Gibbon, M., Spitzer, R.L., Williams, J.B.W., 1996. Structured clinical interview for DSM-IV Axis I Disorders—Patient Version (SCID-I/P). Biometrics Research Department, New York State Psychiatric Institute, New York, NY.

Friis, S., 2010. Early specialised treatment for first-episode psychosis: does it make a difference? Br. J. Psychiatry 196 (5), 339-340.

Gafoor, R., Nitsch, D., McCrone, P., Craig, T.K., Garety, P.A., Power, P., McGuire, P., 2010. Effect of early intervention on 5-year outcome in non-affective psychosis. Br. J. Psychiatry 196 (5),372-376.

Gleeson, J.F., Cotton, S.M., Alvarez-Jimenez, M., Wade, D., Gee, D., Crisp, K., Pearce, T., Newman, B., Spiliotacopoulos, D., Castle, D., McGorry, P.D., 2009. A randomized controlled trial of relapse prevention therapy for first-episode psychosis patients. J. Clin. Psychiatry 70 (4), 477-486.

Gleeson, J.F., Cotton, S.M., Alvarez-Jimenez, M., Wade, D., Gee, D., Crisp, K., Pearce, T., Spiliotacopoulos, D., Newman, B., McGorry, P.D., 2011. A randomized controlled trial of relapse prevention therapy for first-episode psychosis patients: outcome at 30-month follow-up. Schizophr. Bull.

Gleeson, J.F., Lederman, R., Wadley, G., Bendall, S., McGorry, P.D., Alvarez-Jimenez, M., in press. Successful management of safety, confidentiality and privacy within a moderated online social therapy for young people recovering from first-episode psychosis. J. Med. Internet Res.

Goh, T.T., Huang, Y., 2009. Monitoring youth depression risk in Web 2.0. Vine. J. Inf. Knowl. Manag. Syst. 39 (3), 192-202.

Gonzalez-Blanch, C., Crespo-Facorro, B., Alvarez-Jimenez, M., Rodriguez-Sanchez, J.M., Pelayo-Teran, J.M., Perez-Iglesias, R., Vazquez-Barquero, J.L., 2007. Cognitive dimensions in first-episode schizophrenia spectrum disorders. J. Psychiatr. Res. 41 (11), 968-977.

Granholm, E., Ben-Zeev, D., Link, P.C., Bradshaw, K.R., Holden, J.L., 2012. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive textmessaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr. Bull. 38 (3), 414-425.

Hall, P.L., Tarrier, N., 2003. The cognitive-behavioural treatment of low self-esteem in psychotic patients: a pilot study. Behav. Res. Ther. 41 (3), 317-332.

Huang, Y.-P., Goh, T., Liew, C.L., 2007. Hunting Suicide Notes in Web 2.0—Preliminary Findings, Ninth IEEE International Symposium on Multimedia Workshops.

Jackson, H., McGorry, P., Edwards, J., Hulbert, C., Henry, L., Harrigan, S., Dudgeon, P., Francey, S., Maude, D., Cocks, J., Killackey, E., Power, P., 2005. A controlled trial of cognitively oriented psychotherapy for early psychosis (COPE) with four-year follow-up readmission data. Psychol. Med. 35 (9), 1295-1306.

Jackson, H.J., McGorry, P.D., Killackey, E., Bendall, S., Allott, K., Dudgeon, P., Gleeson, J., Johnson, T., Harrigan, S., 2007. Acute-phase and 1-year follow-up results of a randomized controlled trial of CBT versus befriending for first-episode psychosis: the ACE project. Psychol. Med. 1-11.

Jones, R.B., Atkinson, J.M., Coia, D.A., Paterson, L., Morton, A.R., McKenna, K., Craig, N., Morrison, J., Gilmour, W.H., 2001. Randomised trial of personalised computer based information for patients with schizophrenia. BMJ 322 (7290), 835-840.

Killackey, E., Jackson, H.J., McGorry, P.D., 2008. Vocational intervention in first-episode psychosis: individual placement and support v. treatment as usual. Br. J. Psychiatry 193 (2), 114-120.

Kreyenbuhl, J., Nossel, I.R., Dixon, L.B., 2009. Disengagement from mental health treatment among individuals with schizophrenia and strategies for facilitating connections to care: a review of the literature. Schizophr. Bull. 35 (4), 696-703.

Lederman, R., Wadley, G., Gleeson, J.F., Spiteri-Staines, A., Alvarez-Jimenez, M., 2011. Supporting young people with psychosis in the community: an ICT-enabled relapse prevention tool. Proceedings of the Pacific Asia Conference on Information Systems (PACIS), Brisbane.

Lee Duckworth, A., Steen, T.A., Seligman, M.E., 2005. Positive psychology in clinical practice. Annu. Rev. Clin. Psychol. 1, 629-651.

Lieberman, M.A., Goldstein, B.A., 2006. Not all negative emotions are equal: the role of emotional expression in online support groups for women with breast cancer. Psychooncology 15 (2), 160-168.

Linszen, D., Dingemans, P., Lenior, M., 2001. Early intervention and a five year follow up in young adults with a short duration of untreated psychosis: ethical implications. Schizophr. Res. 51 (1), 55-61.

Lloyd, R., Bill, A., 2004. Australia Online: How Australians Are Using Computers And the Internet. 2001 Australian Census Analytic Program. Publication 2056.0. Australian Bureau of Statistics, Canberra.

Lukoff, D., Liberman, R.P., Nuechterlein, K.H., 1986. Symptom monitoring in the rehabilitation of schizophrenic patients. Schizophr. Bull. 12 (4), 578-593.

McFarlane, W.R., 2002. Multifamily Groups in the Treatment of Severe Psychiatric Disorders. Guilford Press, New York.

McFarlane, W.R., Lukens, E., Link, B., Dushay, R., Deakins, S.A., Newmark, M., Dunne, E.J., Horen, B., Toran, J., 1995. Multiple-family groups and psychoeducation in the treatment of schizophrenia. Arch. Gen. Psychiatry 52 (8), 679-687.

McGorry, P.D., Yung, A.R., 2003. Early intervention in psychosis: an overdue reform. Aust. N. Z.J. Psychiatry 37 (4), 393-398.

McGorry, P.D., Edwards, J., Mihalopoulos, C., Harrigan, S.M., Jackson, H.J., 1996. EPPIC: an evolving system of early detection and optimal management. Schizophr. Bull. 22 (2), 305-326.

McGorry, P.D., Killackey, E., Yung, A., 2008. Early intervention in psychosis: concepts, evidence and future directions. World Psychiatry 7 (3), 148-156.

Melle, I., Larsen, T.K., Haahr, U., Friis, S., Johannessen, J.O., Opjordsmoen, S., Simonsen, E., Rund, B.R., Vaglum, P., McGlashan, T., 2004. Reducing the duration of untreated first-episode psychosis: effects on clinical presentation. Arch. Gen. Psychiatry 61 (2), 143-150.

Mohr, D.C., Cuijpers, P., Lehman, K., 2011. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J. Med. Internet Res. 13 (1), e30.

Mueser, K.T., Drake, R.E., 2005. How does a practice become evidenced-based? In: Drake, R.E., Merrens, M.R., Lynde, D.W. (Eds.), Evidence-Based Mental Health Practice: a Textbook. W.W. Norton Company Ltd, New York, pp. 217-241.

Norman, R.M., Malla, A.K., 2001. Duration of untreated psychosis: a critical examination of the concept and its importance. Psychol. Med. 31 (3), 381-400.

Norman, R.M., Malla, A.K., Manchanda, R., Harricharan, R., Takhar, J., Northcott, S., 2005. Social support and three-year symptom and admission outcomes for first episode psychosis. Schizophr. Res. 80 (2-3), 227-234.

Norman, R.M., Manchanda, R., Malla, A.K., Windell, D., Harricharan, R., Northcott, S., 2011. Symptom and functional outcomes for a 5 year early intervention program for psychoses. Schizophr. Res. 129 (2-3), 111-115.

OWASP, the Open Web Application Security Project, t. www.akamai.com.

Penn, D.L., Uzenoff, S.R., Perkins, D., Mueser, K.T., Hamer, R., Waldheter, E., Saade, S., Cook, L., 2010. A pilot investigation of the Graduated Recovery Intervention Program (GRIP) for first episode psychosis. Schizophr. Res. 125 (2-3), 247-256.

Petersen, L., Jeppesen, P., Thorup, A., Abel, M.B., Ohlenschlaeger, J., Christensen, T.O., Krarup, G., Jorgensen, P., Nordentoft, M., 2005. A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. BMJ 331 (7517), 602.

Rashid, T., Ostermann, R.F., 2009. Strength-based assessment in clinical practice. J. Clin. Psychol. 65 (5), 488-498.

Riessman, F., 1965. The ‘helper-therapy' principle. Soc. Work. 10, 27-32.

Rotondi, A.J., Anderson, C.M., Haas, G.L., Eack, S.M., Spring, M.B., Ganguli, R., Newhill, C., Rosenstock, J., 2010. Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr. Serv. 61 (11), 1099-1105.

Ryle, A., Kerr, I.B., 2003. Cognitive analytic therapy. Br. J. Psychiatry 183, 79.

Schuler, D., Namioka, A., 1993. Participatory Design: Principles and Practices. Routledge, Taylor & Francis group, Hillsdale, New Jersey.

Seligman, M.E., Rashid, T., Parks, A.C., 2006. Posit. Psychother. Am. Psychol. 61 (8), 774-788.

Sharkey, S., Jones, R., Smithson, J., Hewis, E., Emmens, T., Ford, T., Owens, C., 2011. Ethical practice in internet research involving vulnerable people: lessons from a selfharm discussion forum study (SharpTalk). J. Med. Ethics 37 (12), 752-758.

Singh, S.P., 2010. Early intervention in psychosis. Br. J. Psychiatry 196 (5), 343-345.

Spaniel, F., Vohlidka, P., Hrdlicka, J., Kozeny, J., Novak, T., Motlova, L., Cermak, J., Bednarik, J., Novak, D., Hoschl, C., 2008. ITAREPS: information technology aided relapse prevention programme in schizophrenia. Schizophr. Res. 98 (1-3), 312-317.

Upthegrove, R., Birchwood, M., Ross, K., Brunett, K., McCollum, R., Jones, L., 2010. The evolution of depression and suicidality in first episode psychosis. Acta Psychiatr. Scand. 122 (3), 211-218.

Valimaki, M., Anttila, M., Hatonen, H., Koivunen, M., Jakobsson, T., Pitkanen, A., Herrala, J., Kuosmanen, L., 2008. Design and development process of patient-centered computer-based support system for patients with schizophrenia spectrum psychosis. Inform. Health Soc. Care 33 (2), 113-123.

Walker, H., 2006. Computer-based education for patients with psychosis. Nurs. Stand. 20 (30), 49-56.

Williams, C., Whitfield, G., 2001. Written and computer-based self-help treatments for depression. Br. Med. Bull. 57, 133-144.

Xia, J., Merinder, L.B., Belgamwar, M.R., 2011. Psychoeducation for schizophrenia. Cochrane Database Syst. Rev. 6, CD002831.

Issues in Mental Health Nursing, 35:323-329, 2014 Copyright ©2014 Informa Healthcare USA, Inc.

ISSN: 0161-2840 print/ 1096-4673 online

DOI: 10.3109/01612840.2013.863412

Problem-solving interventions are not routinely offered to persons with schizophrenia spectrum disorders (SSDs). Telephone calls and text messages are potential avenues to offer problem solving support. This study compared the effect of telephone calls only, text messages only, and both telephone calls and text messages on individuals’ symptoms and medication adherence. Thirty outpatient participants with SSDs were randomly assigned to weekly telephone calls, daily text messages, or both for three months. Participants received monthly in-home pill counts and symptom assessments. Repeated measures ANOVA was significant (F (4,26) =

• 4.2,p = 0.005) for symptom scores. Further work with larger, more diverse, samples is needed.

Schizophrenia spectrum disorders (SSDs) comprise schizophrenia and schizoaffective disorder. The similarities between the disorders include diagnostic overlap (American Psychiatric Association, 2013), symptomatology (Kopelow-icz, Ventura, Liberman, & Mintz, 2008), deficiencies in executive functioning (Premkumar et al., 2008), and associative learning deficits (Sacchetti, Galluzzo, Panariello, Parrinello, & Cappa, 2008). Another commonality is the robust response to problem-solving interventions exhibited by persons with these diseases (Hardeman, Harding, & Narasimhan, 2010; Zygmunt, Olfson, Boyer, & Mechanic, 2002). For the purpose of this project, “problem solving” is defined as the process described by D'Zurrila and Nezu (2006): (1) Identify the problem, (2) Generate/discuss solutions, (3) Select a solution, (4) Plan to implement the solution, and (5) Follow up on effectiveness of chosen solution.

Problem solving has long been among the most effective treatments to improve psychiatric medication adherence; the re-

Address correspondence to Lora Beebe, University of Tennessee,

College of Nursing, 1200 Volunteer Blvd., Knoxville, Tn 37996-4180

USA. E-mail: lbeebe1@utk.edu search evidence supporting this approach spans nearly 30 years (Falloon, McGill, Boyd, Pederson, 1987; Kopelowicz et al., 2012; Penn & Mueser, 1996). Problem solving is a component of many social skills programs for those with SSDs (Liberman et al., 1998) and also is associated with significant improvements in identifying problems and generating solutions when provided alone (Liberman, Eckman, & Marder, 2001).

In spite of their documented effectiveness and the recommendation by experts (Schooler et al., 1997; Zygmunt, Olfson, Boyer, & Mechanic, 2002), problem-solving interventions are not routinely offered to community-dwelling persons with SSDs (Lehman & Steinwachs, 2003). Two significant barriers to the adoption of face-to-face problem-solving programs are labor costs and poor intervention fidelity (Vallina-Fernandez et al., 2001). The most recently published survey examining problemsolving interventions in persons with SSDs concluded that less than half of the participants were receiving any type of problemsolving intervention (Lehman & Steinwachs, 2003). Telephone calls and text messages are potential avenues to meet the need for community-based problem-solving interventions that are accessible to patients at a low cost when compared to face-to-face intervention.

BACKGROUND

Five published studies have examined telephone or texting intervention for persons with schizophrenia spectrum disorders (SSDs). Telephone intervention of up to four months in length was provided via landline in four studies (Beebe, 2001; Beebe & Tian, 2004; Beebe et al., 2008; Montes, Maurino, Diez, & Saiz-Ruiz, 2011) and via cellular telephone in a five-month pilot investigation (Beebe et al., 2010).

Beebe (2001) compared weekly landline telephone intervention for three months following hospital discharge with treatment as usual in 40 persons with SSDs and concluded that this

intervention delivery method is feasible and acceptable to patients. Beebe and Tian (2004) examined responses to landline telephone intervention over six weeks in 24 recently discharged persons with SSDs and reported that persons meeting the telephone intervention provider before calls began conversed significantly longer than controls not meeting the telephone intervention provider during weeks one, two, and three, and were twice as likely to make a feeling statement to the caller. In a follow up study, Beebe and colleagues (2008) compared medication adherence (via monthly, in-home pill count) in 29 persons with SSDs randomly assigned to weekly landline telephone intervention or treatment as usual for three months. Average psychiatric medication adherence was 80% for experimental participants (95% confidence interval [CI] = 67%-88%) and 60.1% (95% CI = 47%-78%) for controls. A repeated measures ANOVA found a statistically significant mean effect for group (F (1,20) = 5.47, p = 0.0298), indicating significantly higher psychiatric medication adherence in those receiving the intervention. Montes and colleagues (2011) compared monthly landline telephone intervention for three months with usual care in 923 outpatient participants with schizophrenia. Adherence was rated by the treating psychiatrist and nurse (who were not blinded) using a four-item scale ranging from 1 (less than 20% of doses) to 4 (more than 80% of doses). Ratings were based upon participant self-report of missed doses and the presence or absence of acute symptoms. These authors (Montes et al., 2011) reported that a significantly higher percentage of patients in the telephone intervention group were classified in group 4 (more than 80% of doses) at the end of the study than controls. The single study examining the feasibility of cellular telephone intervention for persons with schizophrenia (Beebe et al., 2010) reported minor logistical problems, including participants’ difficulties retrieving messages, forgetting to take the telephone when leaving home, and forgetting to charge the telephone.

As a group, these studies demonstrate the feasibility and acceptability of telephone intervention for SSDs and provide preliminary evidence of telephone intervention’s effect upon medication adherence. The studies ranged from six weeks to five months in length, with an average retention rate of 84.75% and an average call completion rate of 73%. This body of work consistently demonstrates the feasibility and acceptability of this delivery method to patients, and documents statistically significant improvement in psychiatric medication adherence over usual care (Beebe et al., 2008).

We located a single published study exploring texting interventions for persons with SSDs. Granholm et al. (2012) provided daily text messages targeting medication adherence, socialization, and auditory hallucinations to 55 community-dwelling persons with SSDs (44 with schizophrenia) for 12 weeks. Text message interventions were based on cognitive behavioral therapy principles (Beck & Rector, 2000). Intervention goals were to provide reminders of health behaviors (e.g., taking prescribed medications); to encourage participants to question unhelpful beliefs, and to assist participants in creating personalized coping plans. The sample consisted mostly of Caucasian (74%) males (69%) and 76% of participants completed the 12-week study. Non-completers scored significantly lower on self-reported living skills and verbal IQ and had higher negative symptom scores than completers. Completers and non-completers did not differ on age, education, positive symptom scores, or depression. Selfreported medication adherence significantly improved among persons living independently, but not among persons living in supported environments. With a text response rate of 86%, Granholm and colleagues (2012) concluded that text messaging interventions are feasible and effective in SSDs, and recommended further investigation in more diverse samples

The purpose of this study was to compare the effect of telephone call intervention only, text messaging intervention only, and both telephone call and text message intervention, over three months, on symptoms and medication adherence in individuals with SSDs. We hypothesized that participants assigned to receive both telephone calls and texting intervention would have higher medication adherence and lower symptom scores than participants receiving only telephone calls or only the texting interventions.

METHODS

Participants and Recruitment

This study enrolled 30 outpatients with SSDs who were receiving care at a community mental health center (CMHC) in the Southeastern United States. The CMHC is a regional, not-for-profit integrated system providing outpatient services to 650+ patients with SSDs. Participants were contacted by the Principal Investigator (PI) and invited to participate in the study while they were at the CMMC for regularly scheduled treatment appointments. Inclusion criteria were: age between 21-68 years, receiving outpatient care at CMHC, chart diagnosis of schizophrenia (any subtype) or schizoaffective disorder using criteria described in Diagnostic and Statistical Manual for Mental Disorders (American Psychiatric Association, 2013), and English speaking. Exclusion criteria were: chart documentation of mental retardation or developmental delay, hearing loss prohibiting telephone communication, or vision or dexterity problems prohibiting texting.

After signing informed consent and providing demographic data, participants were randomly assigned (using a table of random numbers) to one of three groups: daily text message intervention by the PI, weekly telephone call intervention by a trained nurse other than the PI, or both daily text messages from the PI and weekly telephone calls by a trained nurse other than the PI for three months.

Interventions

Telephone Call Intervention

The weekly telephone calls consisted of an intervention developed by the first author and known as Telephone Intervention

TABLE 1

Weekly TIPS Call Topics

Topic

• 1    Are you taking your medication as prescribed? Have you

missed any doses at all?

• 2   Do you know when your next appointment is scheduled?

• 3   Have you had any (symptom) since we last talked?

(Symptoms specific to each participant were collected at baseline.)

• 4   Have you had any cravings for alcohol or other drugs this

week that you’ve found uncomfortable?

• 5   How have you been getting along with others this week?

• 6   Do you have any questions about anything this week?

• 7    Is there anything else you’d like to talk about today?

Problem Solving for Schizophrenia (TIPS). TIPS is a manu-alized intervention that is used to provide weekly support to outpatients with SSDs, and has been described in detail elsewhere (Beebe, 2005). TIPS addresses the following common difficulties in community dwelling persons with SSDs: taking medication, attending appointments, coping with symptoms, abstaining from alcohol and other drugs, and getting along with others. Two open-ended items provide opportunities to address questions or other problems. See Table 1 for a list of TIPS topics (manual available from first author). Trained nurses guide persons with SSDs through the problem-solving process for any difficulties identified, generating solutions, choosing a solution, and following up on the effectiveness of the solution at the next contact. For example, a client reporting problems with medication adherence due to forgetting doses would be assisted to identify several solutions such as making a reminder sign, identifying a family member or caregiver to provide reminders, or tying medication to some other routine, such as brushing one’s teeth. Clients select the solution they wish to implement; the solution and results are reviewed at the next TIPS contact.

Texting Intervention

Based upon the text messaging multiple choice format used by Granholm et al. (2012), we adapted the TIPS telephone protocol for delivery via text message format with multiple choice responses. Each participant received a daily text message topic from the PI for three months. The text message procedure manual (available from first author) provides a format consisting of six distinct message topics with multiple choice responses; messages repeat every six days. See Table 2 for an overview of text message topics. The PI responded to identified difficulties with participant-specific problem solving.

Combined Telephone Intervention and Texting Intervention

Participants in the combined telephone and texting intervention group received weekly phone calls and daily text messages as described above for the three months of the study.

TABLE 2

Daily Text Message Topics

Topic

• 1    Hello from the texting study. Today’s topic is

medications. Did you take your medications today? You can say Yes, Some, I don’t want to, or I forgot.

• 2    Hello from the texting study. Today’s topic is

appointments. Do you know when your next appointment is scheduled? You can say Yes, No, or I’m not sure.

• 3    Hello from the texting study. Today’s topic is symptoms.

Have you been bothered by any [specific symptom here]? You can say Not at all, A little, or A lot.

• 4    Hello from the texting study. Today’s topic is cravings.

Have you had any cravings for alcohol or other drugs that you’ve found uncomfortable? You can say Not at all, A little, or A lot.

• 5    Hello from the texting study. Today’s topic is social. How

have you been getting along with others? You can say Very well, Pretty well, or Not so well.

• 6    Hello from the texting study. Today’s topic is anything

you have questions about.

• 7   Thank you for being a part of the texting study. We

appreciate you! Have a great day!

Measures

Pill Counts

The PI initiated telephone contact to schedule monthly inhome pill counts. Participants were requested to produce all medications dispensed in pill form to be counted. A measure of adherence was generated by dividing the number of pills missing from the bottle(s) by the number of pills prescribed within the time period of the prescription. Adherence percentage was calculated separately for each oral medication in pill form using the following formula:

/ Number of pills missing from the bottle \ Number of pills prescribed since prescription refill date

As an example, consider this hypothetical participant who filled a medication prescription on July 1 for 30 pills of 100 mg Seroquel, with the instruction to take one by mouth at bedtime. At the pill count visit on July 31 at 3 pm, the blinded research assistant found 25 pills missing from the bottle when a total of 30 pills had been prescribed since the refill date. 25/30 x 100 = 2500/30 = 83.3% adherence for Seroquel over the specified time period for this participant.

For participants prescribed depot medication, adherence for the depot medication was calculated (via record review) as a percentage of injections received of the total number of injections prescribed during each month. It is the clinical practice at

the recruitment site to record depot prescriptions in three-month intervals. As an example, consider this hypothetical participant who was prescribed to receive 50mg of Haldol Decoanate intramuscularly every two weeks from June 1 through August 31. Injection clinics take place once a week on the same day, so the three injection days for July were Monday July 1, 15, and 29. If the participant presented for the two injections scheduled on July 15 and 29, their adherence percentage for Haldol Depot would be 2/3, or 66.6%. All participants receiving depot medication also were receiving oral psychiatric and/or nonpsychiatric medications; procedures for their in-home pill count and oral adherence calculations were identical to those not prescribed depot medication.

If more than one psychiatric or nonpsychiatric medication was prescribed, an overall adherence percentage was calculated by averaging the percentage adherence of all medications within that category (both oral and depot). Overall adherence was calculated as the mean adherence for all medications in a group (psychiatric or nonpsychiatric). For this study, we defined psychiatric medications as typical and atypical antipsychotics, antidepressants, anti-anxiety medications, mood stabilizers, antiparkinsonian medications, and hypnotics. All other medications were defined as nonpsychiatric medications. Procedures for inhome pill counts were identical to those used in our prior work (Beebe et al, 2010).

Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1988)

The BPRS is an 18-item clinician rated scale addressing a broad range of psychopathology. Scores ranged from 0-108 with higher scores indicating greater disturbance. Weighted Kappa coefficients ranged from 0.52-0.90 for individual items with a mean of 0.72 for all items (Gabbard et al., 1987). The measure has good internal consistency with Cronbach’s alphas of 0.81 and 0.91 for positive and negative symptoms, respectively (Nicholson, Chapman, & Neufeld, 1995) and good concurrent validity (Faustman, 1994; Guy, 1976; Hedlund & Vieweg, 1980). The BPRS takes about 30 minutes to administer and score. After reviewing the interview scale/scoring guidelines, the blinded research assistant (RA) and PI practiced scoring the BPRS with simulated patients. Scores were recorded independently; mean inter-rater reliability was 0.89.

Procedures

Potential participants were approached while at the recruitment site for their regularly scheduled treatment appointments. HIPAA law and the Notice of Privacy Practices, signed by all patients at the recruitment site, allow disclosure of Protected Health Information (PHI) for research, authorizing the initial case reviews and communications required to identify potential participants. The PI approached 37 persons meeting the study criteria regarding participation; seven persons declined. Three persons cited a lack of confidence using a cellular telephone as a reason for declining, two reported feeling unable to participate due to their symptom level, and two objected to the time commitment. The 30 persons agreeing were randomly assigned (using a table of random numbers) to weekly TIPS only, daily text messages only, or both weekly TIPS and daily text messages for three months.

Sociodemographic characteristics, living arrangements, educational level, prescribed medications, and BPRS scores were collected at baseline by the PI in a private office at the CMHC immediately following the granting of informed consent but before group assignment. Thus, the PI was blinded as to group assignment when conducting the baseline BPRS. All study participants were provided a basic cellular telephone with unlimited calling and texting for three months. The PI provided daily text messages to participants randomly assigned to receive them and documented responses by hand. A trained nurse (a TIPS provider in two prior studies, Beebe et al., 2008 and Beebe et al., 2010) provided TIPS interventions and documented participant responses by hand. All participants received monthly in-home pill counts and BPRS evaluations for three months by the RA (different than the TIPS nurse). Following each monthly assessment, participants received a $10 Wal-Mart gift card.

RESULTS

The sample consisted of 19 females and 11 males with a mean age of 48.7 years (range: 23-64, SD = 11.6). The sample was equally divided between Caucasian and African American participants and most participants lived alone (n = 16). The most commonly prescribed medications were oral atypical antipsychotics (n = 20) and anti-parkinsonian agents (n = 17). Fifteen participants were prescribed at least one nonpsychiatric medication, most commonly anti-hypertensives (n = 14) (see Table 3).

Twenty-eight participants completed the three-month study. One participant’s telephone was exposed to water on day three; unfortunately, study funds did not allow purchase of replacement telephones. One participant was hospitalized on day 54 and was unable to complete the study due to deteriorating physical condition. Both participants not completing the study were assigned to the text only group. Hence, ten participants in the TIPS only group, ten participants in the TIPS plus text group, and eight participants in the text only group completed the study.

BPRS Scores

There were no statistically significant baseline differences among the three groups on any demographic variables examined; however a one way ANOVA revealed a statistically significant difference in baseline BPRS scores (F (2,27) = 6.1, p = 0.007); the mean baseline BPRS score of those assigned to receive TIPS only was 50.1, the mean baseline BPRS score of the text only group was 41.3, and the mean baseline BPRS score of the TIPS plus text group was 37.7 (see Table 4).

Results of Hypothesis Testing

Three 2 x 3 ANOVA models with one independent measure (Group) and one repeated measure (Time) were calculated to determine differences in psychiatric medication adherence, nonpsychiatric medication adherence, and symptom levels based on intervention type, with percentage adherence to psychiatric medications as the dependent variable in the first model, percentage adherence to nonpsychiatric medications as the dependent variable in the second model, and BPRS scores as the dependent variable in the third model. In our ANOVA models we examined three different variance and covariance structures (compound symmetry, unstructured, and Huynh-Felt) while controlling for differences in baseline BPRS scores. The results of the three analyses were similar; results of the unstructured analysis are reported in this article. These analyses yielded a statistically significant main effect for group (F (4,26) =

• 4.2, p = 0.005) for BPRS scores. Mean BPRS scores of the TIPS plus text group were lower than the text only group (by an average of 9.2 points) at two of three post-intervention measurement points. Mean BPRS scores of the TIPS plus text group were lower than the TIPS only group (by an average of 5.7 points) at all three post-intervention measurement points (see Table 4).

There was no significant Group x Time interaction for psychiatric, F (4,26) = 1.24, p = 0.31) or nonpsychiatric, F (4,26) = 0.53, p = 0.71) medication adherence. Nevertheless, our findings were in the predicted direction for both psychiatric and nonpsychiatric medication adherence: Mean psychiatric adherence scores for the TIPS plus text group were higher than both the TIPS only (by an average of 5.3%) and the text only groups (by an average of 13%) at each of the three postintervention measurement points (see Table 5). Mean nonpsychiatric medication adherence scores for the TIPS plus text group were higher (by an average of 11.9%) than the text only group at two of the three post-intervention measurement points, and higher than the TIPS only group (by an average of 14.9%) at all three post-intervention measurement points (see Table 6). A post hoc analysis revealed that, based upon the sample size examined, the power to examine psychiatric and nonpsychiatric medication adherence was 34% and 25%, respectively.

Additional Analyses

We used Chi square to examine relationships among ordinal variables. Where at least one cell of the chi square had an expected count of less than five, Fischer's exact tests were computed. There were no significant associations among the ordinal variables examined with the exception of living arrangement—significantly more participants in the TIPS only group lived with family (n = 7) than those in the text only (n = 3) or TIPS plus text (n = 1) groups. (Fischer's exact = 9.77, p = 0.02).

DISCUSSION

To our knowledge this is the first study comparing text and telephone intervention for persons with SSDs. Our findings are congruent with the small body of literature documenting increased medication adherence with telephone intervention. Our prior work using identical pill count procedures documented 80% psychiatric medication adherence over three months in those receiving TIPS (Beebe et al., 2008); similarly, the average psychiatric medication adherence over this three-month study was 84.3% in the TIPS plus text group and 78.9% in the TIPS only group. Likewise, Montes et al. (2011) reported that significantly more participants receiving telephone intervention took at least 80% of their prescribed medication doses (ratings based upon self-report and symptom ratings).

Our finding of nonsignificant differences in medication adherence between TIPS only, text only, and TIPS plus text participants requires further elaboration. A possible explanation is that differences in the groups were small due to the lack of a control condition of usual care. In our prior study, which included a treatment as usual control group, those assigned to the control condition had an average psychiatric medication adherence of 60.1% over three months (Beebe et al., 2008). Furthermore, eight clients in our sample (40%) were receiving services from the Program for Assertive Community Treatment (PACT), a 24hour, intensive treatment delivery system for high service users. PACT clients have their medication delivered daily by treatment staff, resulting in very high medication adherence, and requiring a larger sample to detect the small adherence increases observed. Psychiatric medication adherence in the eight participants assigned to PACT was 96.7% in month 1, 96.2% in month 2, and 98% in month 3.

We located no other published work examining symptom levels of persons with SSDs receiving telephone intervention. Our finding of significantly lower BPRS scores, even when baseline scores were controlled for, provides preliminary evidence for the efficacy of TIPS in symptom reduction, particularly when combined with daily text messages. Mean BPRS scores of the TIPS plus text group were lower than the text only group (by an average of 9.2 points) at two of three postintervention measurement points. Mean BPRS scores of the TIPS plus text group were lower than the TIPS only group (by an average of 5.7 points) at all three post-intervention measurement points.

LIMITATIONS

The use of a small, convenience sample recruited from a single community mental health center limits the generalizability of our findings. The smaller than expected increases in medication adherence resulted in power less than 35% to detect significant differences in adherence with the current sample size, introducing the possibility of Type II error. Finally, although pill counts are a commonly used proxy measure for medication adherence, pill counts do not guarantee that missing medications were actually ingested (Beebe et al., 2008).

The inclusion of participants receiving depot medications could have increased our adherence rates, since depot formulations are associated with approximately 10% greater adherence than oral psychiatric medications (Zoler, 2012). Fifteen participants were receiving depot medications in addition to other oral psychiatric medications: seven in the TIPS only group; five in the text only group; and three in the TIPS plus text group. In spite of this limitation, the inclusion of persons receiving depot medications results in a final sample that may more accurately represent the target population for the TIPS intervention.

CONCLUSION

This study provides preliminary evidence for the superiority of telephone intervention plus text messages over either telephone or text intervention alone in symptom reduction and medication adherence increases in persons with SSDs. Our findings indicate that adding text messages to the TIPS intervention provides small gains in both symptom reductions and increased adherence to psychiatric and nonpsychiatric medications. Further, it appears that text messages are a feasible adjunct intervention in SSDs. More research with larger, more diverse samples is needed to refine text messaging procedures, to identify subgroups for which this intervention might be optimally useful, and to examine the intervention’s effects over longer time periods. Our follow-up study will examine the correlations between in-home pill count and serum medication levels over nine months.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

REFERENCES

American Psychiatric Association. (2013). Diagnostic andstatistical manual of mental disorders (5th ed.). Washington, DC: Author.

Beck, A. T., & Rector, N. A. (2000). Cognitive therapy of schizophrenia: A new therapy for the new millennium. American Journal of Psychotherapy, 54, 291-300.

Beebe, L. H. (2001). Community nursing support for persons with schizophrenia. Archives of Psychiatric Nursing, 15(5), 214-222.

Beebe, L. H. (2005). Telephone Intervention-Problem Solving (TIPS) for Persons with schizophrenia. Directions in Psychiatric Nursing, 11(9), 103-112

Beebe, L. H., Smith, K., Bennett, C. et al. (2010). Keeping in touch: Cellular telephone use in persons with schizophrenia spectrum disorders. Journal of Psychosocial Nursing and Mental Health Services, 48(4), 32-37.

Beebe, L. H., Smith, K., Crye, C. et al. (2008). Telenursing intervention increases psychiatric medication adherence in schizophrenia outpatients. Journal of the American Psychiatric Nurses Association, 14(3), 217-224.

Beebe, L. H., & Tian, L. (2004). TIPS: Telephone Intervention-Problem Solving for persons with schizophrenia. Issues in Mental Health Nursing, 25(3), 317-329.

D’Zurrila, T. D., & Nezu. A. M. (2006). Problem-solving therapy: A positive approach to clinical intervention (3rd ed.). New York, NY: Springer.

Falloon, I. R., McGill, C. W., Boyd, J. L., & Pederson, J. (1987). Family management in the prevention of morbidity of schizophrenia: Social outcome of a two-year longitudinal study. Psychological Medicine, 17(1), 59-66.

Faustman, W. O. (1994). Brief psychiatric rating scale. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (pp. 371-401). Hillsdale, NJ: Erlbaum.

Gabbard, G. O., Kennedy, L. L., Deering, C. D. et al. (1987). Inter-rater reliability in the use of the brief psychiatric rating scale. Bulletin of the Menninger Clinic, 51, 519-531.

Granholm, E., Ben-Zeev, D., Link, P. C., Bradshaw, K. R., & Holden, J. L. (2012). Mobile Assessment and Treatment for Schizophrenia (MATS): A pilot trial of an interactive text-messaging intervention for medication adherence, socialization and auditoryhallucinations. SchizophreniaBulletin, 38(3), 414-425.

Guy, W. (1976). Assessment manual for psychopharmacology revised. DHEW pub # ADM 76-338. Rockville MD, US Department of Health Education and Welfare, Public Health Service, Alcohol, Drug Abuse and Mental Health Administration, National Institute of Mental Health Psychopharmacology Research Branch, Division of Extramural Research Programs.

Hardeman, S. M., Harding, R. K., & Narasimhan, M. (2010). Simplifying adherence in schizophrenia. Psychiatric Services, 61(4), 405-408.

Hedlund, J. L., & Vieweg, B. W. (1980). The brief psychiatric rating scale (BPRS): A comprehensive review. Journal of Operational Psychiatry, 11, 48-65.

Kopelowicz, A., Ventura, J., Liberman, R. P., & Mintz, J. (2008). Consistency of brief psychiatric rating scale factor structure across a broad spectrum of schizophrenia patients. Psychopathology, 41(2), 77-84.

Kopelowicz, A., Zarate, R., Wallace, C. J., Liberman, R. P., Lopez, S. R., & Mintz, J. (2012). The ability of multifamily groups to improve treatment adherence in Mexican Americans with schizophrenia. Archives of General Psychiatry, 69(3), 265-273.

Lehman, A. F., & Steinwachs, D. M. (2003). Evidence-based psychosocial treatment practices in schizophrenia: Lessons from the patient outcomes research team (PORT) project. Journal of the American Academy of Psychoanalytic and Dynamic Psychiatry, 31(1), 141-154.

Liberman, R. P., Eckman, T. A., & Marder, S. R. (2001). Rehab rounds: Training in social problem solving among persons with schizophrenia. Psychiatric Services, 52(1), 31-33.

Liberman, R. P., Wallace, C. J., Blackwell, G., Kopelowicz, A., Vaccaro, J. V., & Mintz, J. (1998). Skills training versus psychosocial occupational therapy for persons with persistent schizophrenia. American Journal of Psychiatry, 155(8), 1087-1091.

Montes, J. M., Maurino, J., Diez, T., & Saiz-Ruiz, J. (2011). Factors associated with the effectiveness of a telephone based nursing strategy for enhancing medication adherence in schizophrenia. Clinical Practice in Epidemiology and Mental Health, 7, 117-119.

Nicholson, I. R., Chapman, J. E., & Neufeld, R. W. J. (1995). Variability in BPRS definitions of positive and negative symptoms. Schizophrenia Research, 17, 177-185.

Overall, J. E., & Gorham, D. R. (1988). The Brief Psychiatric Rating Scale (BPRS) recent developments in ascertainment and scaling. Psychopharma-cological Bulletin, 24, 97-99.

Penn, D. L., & Mueser, K. T. (1996). Research update on the psychosocial treatment of schizophrenia. American Journal of Psychiatry, 153(5), 607-617.

Premkumar, P., Cooke, M. A., Fannon, D., Peters, E., Michel, T. M., Aasen, I., Murray, R. M., Kuipers, E., & Kumari, V. (2008). Misattribution bias of threat-related facial expressions is related to a longer duration of illness and poor executive function in schizophrenia and schizoaffective disorder. European Psychiatry, 23(1), 14-19.

Sacchetti, E., Galluzzo, A., Panariello, A., Parrinello, G., & Cappa, S. F. (2008). Self-ordered pointing and visual conditioning associated learning tasks in drug free schizophrenia spectrum disorder patients. BMC Psychiatry, 23(8), 6.

Schooler, N. R., Keith, S. J., Severe, J. B. et al. (1997). Relapse and rehospitalization during maintenance treatment of schizophrenia. The effects of dose reduction and family treatment. Archives of General Psychiatry, 54(5), 453-463.

Vallina-Fernandez, O., Lemos-Giraldez, S., Roder, V. et al. (2001). Rehab rounds: An integrated psychological treatment program for schizophrenia. Psychiatric Services, 52(9), 1165-1167.

Zoler, M. (2012, May 23). Depot injection boosts antipsychotic compliance in schizophrenia. Clinical Psychiatry News. Retrieved from http://www.clinicalpsychiatrynews.com

Zygmunt, A., Olfson, M., Boyer, C. A., & Mechanic, D. (2002). Interventions to improve medication adherence in schizophrenia. American Journal of Psychiatry, 159(10), 1653-1664.

Contents lists available at ScienceDirect

Schizophrenia Research

journal homepage: www.elsevier.com/locate/schres

Pilot randomised controlled trial of a brief coping-focused intervention for hearing voices blended with smartphone-based ecological momentary assessment and intervention (SAVVy): Feasibility, acceptability and preliminary clinical outcomes

Imogen H. Bell ^a, ², Susan L. Rossell ^a,^b, John Farhall ^c,^d, Mark Hayward ^e,^f, Michelle H. Lim ^a, Sarah F. Fielding-Smith ^e,^f, Neil Thomas ^a

^a Centre for Mental Health, Swinburne University of Technology, Australia

^b Department of Psychiatry, St. Vincent's Hospital, Australia

^c Department of Psychology and Counselling, La Trobe University, Australia

^d NorthWestern Mental Health, Melbourne Health, Australia

^e Sussex Partnership NHS Foundation Trust, UK

^f School of Psychology, University of Sussex, UK

Background: Voice-hearing experiences can be distressing and impairing, and existing psychological treatments show modest effectiveness. Ecological momentary assessment and intervention (EMA/I) are two promising approaches which may be used as digital tools to support and enhance existing psychological therapies. The aim of this study was to investigate the potential clinical utility of smartphonebased EMA/I in a blended, coping focused therapy for voice-hearing experiences. Method: This pilot RCT focused on feasibility, acceptability and preliminary estimations of efficacy. Thirty-four participants with persisting and distressing voices were randomised to receive the four-session intervention along-side treatment-as-usual (TAU) or TAU-only. Results: Findings supported the feasibility and acceptability of the approach, with good engagement and satisfaction rates, and clinical outcomes showed the intervention holds promise for improving coping, overall severity of voices and to some degree their negative impact. Conclusion: This is the first examination of the use of EMA/I in a blended therapy for psychotic experiences, with findings suggesting these technologies show promise as clinical tools.

© 2019 Elsevier B.V. All rights reserved.

1. Introduction

Psychological interventions, particularly cognitive behavioural therapy for psychosis (CBTp; Farhall and Thomas, 2013; Turkington et al., 2008), are recommended as a core component of treatment for hearing voices, or auditory verbal hallucinations (Galletly et al., 2016; NICE, 2010). However, effects sizes of CBTp are modest (Jauhar et al., 2014; van der Gaag et al., 2014), access is generally limited, and the treatment is costly and complex to deliver (Berry and Haddock, 2008; Schizophrenia Commission, 2012; Haddock et al., 2014; Ince et al., 2016).

Smartphones applications (apps) have unique capabilities that may provide novel and innovative ways to improve the effectiveness and reach of psychological treatments for psychosis (Bakker et al., 2016; Price et al., 2014; Proudfoot, 2013; Thomas et al., 2019; Treisman et al., 2016). Ownership of smartphones is wide spread in psychosis populations (Firth et al., 2015) and research has shown that these technologies can assist in illness selfmanagement, reduce symptoms and their impact, minimise relapse, and promote physical health (Alvarez-Jimenez et al., 2014; Bell et al., 2017; Firth and Torous, 2015). Further, individuals with psychosis appear interested in using them regularly for their mental health (Bucci et al., 2018b; Gay et al., 2016; Torous et al.,

• 2014).

Ecological momentary interventions (EMIs; Heron and Smyth,

• 2010) make use of smartphone technology to provide support in daily life by sending electronic prompts that encourage therapeutic behaviours in the moment they are needed. When used in conjunction with standard therapy, EMI might promote generalisation of intervention strategies to daily life. This is a potentially important capability for the treatment of psychosis, where cognitive and motivational difficulties are common (Fioravanti et al., 2012; Johansen et al., 2011) and treatment engagement can be low (Dixon et al., 2016; Johansen et al., 2011). Studies have demonstrated the broad application of EMI in psychosis treatment, with support for feasibility, acceptability and promise for improving clinical outcomes (Bell et al., 2017).

Ecological momentary assessment (EMA; Shiffman et al., 2008) is a related approach which uses smartphone apps to deliver questionnaires in daily life at repeated intervals across several days. Greater reliability and ecological validity are afforded through measurement of phenomenon in the moment, in natural environments (Shiffman et al., 2008; Stone et al., 2007). Further, repeated measurement allows for the examination of temporal relationships between variables (Ebner-Priemer and Trull, 2009). Findings have supported the feasibility and reliability of using EMA in psychosis populations (Brenner and Ben-Zeev, 2014; Granholm et al., 2008; Palmier-Claus et al., 2012), however little research has investigated its use in clinical treatment (Bell et al., 2017; McDevitt-Murphy, Luciano and Zakarian, 2018).

EMA and EMI (EMA/I) may be useful within blended therapies for psychosis, involving the combined used of digital technologies with standard face-to-face therapies. In other clinical populations, studies have shown that these approaches may lead to more potent interventions (Erbe et al., 2017). It may be possible to adapt existing evidence-based treatments to incorporate technologies such as EMA/I to support therapeutic components. In the context of voicehearing experiences, one such candidate therapy is coping strategy enhancement (CSE; Tarrier, 1992; Tarrier et al., 1990).

CSE is an idiographic, CBT-based psychological therapy aiming to improve coping with psychotic symptoms (Tarrier, 1992; Tarrier et al., 1990). Functional analysis is used to identify antecedents and responses to symptoms, which then informs the identification and subsequent implementation of individualised coping strategies. Trials have supported the clinical benefit of CSE approaches (Tarrier et al., 1993,1998; Yusupoff and Tarrier, 1996), including in a brief format over four sessions targeting voices specifically (Hayward et al., 2018; Paulik et al., 2018).

Conceivably, such an approach may be enhanced by incorporating EMA to assist in initial functional analysis, providing data on variation in voices and related variables, and EMI prompts of individualised coping strategies may promote more consistent use of these in daily life. This highly novel application of EMA/I was the subject of this research, which aimed to examine the feasibility, acceptability and estimated clinical effects of a brief intervention which blended EMA/I with standard face-to-face therapy to improve coping with hearing voices [Smartphone-Assisted coping focused interVention for Voices (SAVVy)]. The development of the intervention and a case illustration is reported in Bell et al. (2018a).

2. Methods

Reporting followed CONSORT guidelines (supplementary) and ethical approval was provided by the Alfred Hospital Ethics Committee (project 440/16). The trial was prospectively registered (ACTRN12617000348358) and the study protocol was published before recruitment ended (Bell et al., 2018b).

2.1. Study design

A single-blind, parallel group, pilot RCT with a 1:1 allocation ratio to the SAVVy intervention plus treatment-as-usual (TAU) or TAU alone. TAU typically included standard care provided by a clinical team, including medication and case management. Trained researchers blind to treatment allocation completed assessments at pre-randomisation and approximately 8 weeks following randomisation. An independent researcher randomly allocated participants to groups used minimisation procedure using QMinim online software. Minimisation was used to balance continuous vs non-continuous voices across groups [Psychotic Symptom Rating Scales-Auditory Hallucinations (PSYRATS-AH; Haddock et al., 1999) item 1 score <3 versus 4], a variable which may be influenced by EMA/I.

2.2. Participants

Thirty-four adult participants were recruited through referrals to a specialist Voices Clinic and wider advertising to clinical services and consumer groups, between March 2017 and January 2018. The sample size was based on published guidelines (Julious, 2005; Sim and Lewis, 2012; Teare et al., 2014) and is consistent with similar recent pilot RCTs with this population (Bucci et al., 2018; Hazell et al., 2018). Eligibility criteria were: (1) over the age of 18 years; (2) proficient English language; (3) experiencing current, frequent (4 + times per week, or if less, lasting at least 1 h at a time) and distressing (score 1 + on amount of distress item of PSYRATS-AH (Haddock et al., 1999) voices for at least six months; (4) comfortable using a smartphone or willing to learn. Exclusion criteria were (1) unable to provide informed consent; (2) intellectual disability (estimated IQ< 70, measured by the Wechsler Test of Adult Reading (WTAR; Wechsler, 2001); (3) initiation of a new antipsychotic medication within the previous 8 weeks; (4) voices solely substance-related; (5) distress or agitation displayed during baseline assessment; and (6) requiring active crisis management.

2.3. Measures

Baseline measures included basic demographic and clinical information (e.g. medication dosages); use and familiarity with technology; Mini-International Neuropsychiatric Interview (Sheehan et al., 1998) for DSM-5 mental disorders; Structured Clinical Interview for DSM-5 (First et al., 2015) for borderline personality disorder diagnosis; Scale for the Assessment of Negative Symptoms (SANS; Andreasen, 1989); and Wechsler Test of Adult Reading (WTAR; Wechsler, 2001) to estimate intellectual ability.

Feasibility was the primary outcome measure, focusing on: completion rates of the EMA questionnaires (completers defined as having completed over 33% of the total number of EMA questionnaires), proportion of participants for whom EMA-based feedback summaries were produced; proportion of EMI reminders viewed; trial uptake and attrition; and fidelity to the intervention protocol (proportion of therapy checklist items endorsed by therapists as completed).

Acceptability was measured using a feedback questionnaire designed for the study. Participants completed the Credibility and Expectancy Questionnaire (CEQ; Devilly and Borkovec, 2000) after the informed consent procedure, of which the Credibility subscale was used to measure pre-conceived perceptions of the intervention credibility. Treatment group participants completed the Working Alliance Inventory — Short Revised (WAI-SR; Hatcher and Gillaspy, 2006) in relation to their rapport with the therapist.

The primary clinical outcome was PSYRATS-AH total score (Haddock et al., 1999). Secondary clinical outcomes included Depression, Anxiety and Stress Scale-21 total score (DASS-21; Lovibond and Lovibond, 1995) and the Subjective Experiences of Psychosis Scale —Negative Impact Subscale total score (SEPS; Gillian Haddock et al., 2011). Process measures included two visual analogue scales (VAS) to assess (1) confidence in coping with voices

I.H. Bell et al. / Schizophrenia Research xxx (xxxx) xxx                                                              3

day-to-day, and (2) awareness of patterns in voices, and two multiple choice items measuring the frequency of use of coping strategies and the number of strategies used.

2.4. Intervention

Details of the development and a case study illustrating the delivery of the intervention are reported elsewhere (Bell et al., 2018a). A depiction of the intervention procedure is displayed in Fig. 1. The intervention was split into two phases involving initial assessment and EMA monitoring for functional analysis, which informed the second phase involving identifying and implementing individualised coping strategies which were supported by personalised EMI reminders in daily life. An existing app called MovisensXS was used for the purpose of the trial. Participants were lent a smartphone if they did not already have a compatible Android phone (necessary to run the app).

Following the first session involving an introduction and training in how to use the smartphone app, participants completed six days of EMA monitoring involving the completion of a 39-item questionnaire (supplementary 2), ten times per day. EMA items measured common antecedents to voices (e.g. mood, anxiety), voice-related variables (i.e. intensity, distress, impact), and coping responses to the voices (e.g. listening to music, arguing with the voices). EMA items were determined based on an iterative approach involving reviews of the EMA literature on psychotic symptoms and coping with voices, feedback from lived experience consultants, researchers and clinicians in the field, and an initial pilot study (Bell et al., 2018a). This EMA period facilitated selfmonitoring of voices, and provided data that was then statistically analysed using time-lagged multiple regressions to identify variables associated with fluctuations in voice intensity (see Bell et al., 2018a, 2018b). Summary statistics of questionnaires completion rates, voice intensity and distress, and coping strategies used were also computed. In line with recommendations in the literature, this feedback analysis was conducted if participants completed at least 33% of the total number of EMA surveys (Delespaul, 1995; Palmier-Claus et al., 2011). A simple, lay-person summary of this analysis was provided to participants in the second session, which was discussed in an exploratory manner to inform functional analysis of the voices. This functional analysis was used to identify alternative responses to the voices which may interrupt problematic maintenance cycles associated with their activity and improve overall coping. Individual coping strategies were worded as short sentences by the participant and programmed into the app. Participants then received five personalised EMI prompts per day for the following ten days after session two and were able to view their reminders on-demand. Eight evening EMA questions were used to monitor changes in the voices and helpfulness of the coping strategies, with feedback of this information then reviewed in session three and coping strategies could be updated if needed. This was followed by a further ten-day EMI period and evening EMA questions, with the final session involving a review and ending of the intervention.

2.5. Analysis

Feasibility and acceptability results are reported descriptively. Clinical outcomes were analysed on an intention-to-treat basis using all available data. Missing cases were treated as missing at random with a small number of individual missing data points (<5%) imputed using an expectation-maximization method and multiple imputation used for the three missing outcome cases (pooled aggregate of 50 iterations; Enders, 2001; Tabachnick and

Fidell, 2007). Clinical outcomes are presented as pooled means and standard deviations, with Hedges' g formula used to calculate standardised effect sizes and associated 95% confidence intervals (Durlak, 2009). Effect sizes for all variables were coded so that positive values reflected changes favouring the treatment group. In line with our planned analysis, analysis of covariance (ANCOVA) was used to test for differences between treatment and control group scores on all clinical outcome and process measures at the outcome timepoint, controlling for baseline scores. For the two ordinal measures of frequency and number of coping strategies used, Wilcoxon Sign Ranks Test were used to compare pre-post changes within each group. All outcome analyses were conducted using IBM SPSS Statistics Version 25 (SPSS Inc, 2017). EMA and EMI feasibility statistics and EMA-derived feedback analysis within the intervention were conducted using Stata Version 14.1 (StataCorp, 2015).

3. Results

3.1. Sample characteristics

Demographic and clinical characteristics of both groups are provided in Table 1. Group comparisons revealed significantly higher SANS scores in the control compared to treatment group (F(1,32) = 7.86, p < .01), with no other differences identified.

3.2. Feasibility

Fig. 2 displays the CONSORT flow diagram. Of those screened for eligibility, there was a 34% uptake into the trial. Data were available for 31 (91%) participants at post-treatment, with 3 participants lost to follow-up (2 control, 1 treatment) and no withdrawals. Of the 17 participants in the treatment group, 13 completed all four sessions.

Two stated they could no longer find time for participation (one attending three sessions and the other one session), and the two remaining participants discontinued sessions due to worsening in mental health precluding their ability to attend appointments (one attending three sessions and the other one session). These reasons were judged as factors external to the trial and unrelated to the intervention. Group comparisons did not reveal any significant differences between those who did and did not drop out on any demographic or clinical variables, or confidence using smartphones (p > .05). Ninety percent of therapy checklist items were endorsed as completed across the four sessions.

Two participants discontinued therapy during the EMA monitoring period, and another experienced a technological issue with the smartphone alerts, and was therefore excluded from EMA/I completion analysis. Across the remaining 14 participants, the average completion rate of the daytime EMA questionnaires was 72%. One-hundred percent of participants reached the minimum 33% completion rate criteria necessary to produce the EMA-derived feedback. Unplanned Pearson's correlation analyses did not reveal any significant relationships (p > .05) between completion rates and confidence in using smartphone apps, demographic or clinical variables, or clinical outcome variables. Completion rates of evening EMA questionnaires across the intervention was 74%. Scheduled EMI reminders were viewed on average 2.5 times per day, and

• 1.5 times per day when user-initiated (one participant was excluded due to technical issues resulting in additional EMI reminders).

3.3. Acceptability

Table 2 displays the average responses to each item of the satisfaction questionnaire. Overall, responses reflected good satisfaction across all elements of the intervention, with 100% of treatment group participants agreeing that they would recommend it to other people who hear voices. Open feedback was minimal, but largely positive, with all verbatim responses displayed in Table 3. The average WAI-SR item score (range 1—5) was 4.33 (SD = 0.55), suggesting that participants developed positive working alliances with the therapist. The average CEQ scores of the 3-item Credibility subscale (range 1—9) were 7.5 (S = 1.56) for the perceived logic of the therapy, 6.68 (SD = 1.53) for the perceived success of the therapy, and 6.56 (SD = 2.21) for the confidence in recommending the therapy to others. Mean scores on the CEQ and WAI-SR were similar to those reported in other trials in analogous populations (Gaudiano et al., 2015; Webb et al., 2013; White et al., 2011).

3.4. Clinical outcomes

Scores on all clinical outcome and process measures for both groups at baseline and outcome timepoints are displayed in Table 4. Moderate effect sizes favouring the treatment group were observed for PSYRATS-AH total score with a trend towards significant difference between groups (F(1,31) = 3.00, p = .09). Small, nonsignificant effects favouring the treatment group for the SEPS negative impact subscale (F(1,31) = 0.55, p = .46), and very small, non-significant effects favouring the control group on DASS-21 scores (F(1,31) = 1.87, p = .18), were observed. A very large effect favouring the treatment group was observed for the VAS coping with voices item and differences between groups were significant (F(1,31) = 23.59, p < .001). Similarly, the VAS awareness of patterns in voices item was also significantly different between groups (F(1,31) = 5.40, p < .05), with a moderate effect size. Analyses with significant group differences were run again with SANS as an additional covariate. Group differences for the VAS coping with

I.H. Bell et al. / Schizophrenia Research xxx (xxxx) xxx

Table 2

Means and standard deviations of item responses to the satisfaction survey.

Table 3

Verbatim open feedback from participants.

Negative                                                                         Positive

Just the problems with the phone and the app in the second stage. Became a bit Thanks to this study I now realise I can have a better and more happy, stress-free and tedious.^a                                                                      peaceful life if I can take a good look objectively at the experience of the voices and

The initial number of prompts in the first week was a little annoying.              work out some constructive ways of dealing with them.

It required a lot of time and effort. Which is okay but its been a busy time of the year It helped me to control my voices and to make me feel better about myself.

for the first time in many years and I found it hard to focus on everything and keep It helped me to make a connection between the voices and my own thoughts and up.^b                                                                           feelings. It reminded me to take care of myself.

Discussing what I am talking, getting confused at times about my past.            Identifying triggers and patterns was helpful in terms of understanding my voices,

When it beeped at inappropriate times, middle of writing an email or at church. how to cope with them, and the discussion I had with the therapist.

I could talk about the voices and they could understand how I was feeling. Collaboratively exploring new ways to understand my negative self-sabotaging voice and how it can and does affect me in daily activities.

Good to discuss what I experience with someone who understands.

I tended to be able to notice when the voices were getting worse or better. A good reminder about time passing, to eat, do something.

Detecting patterns in the voices.

There was nothing I didn't like, it was all good and I had the support to cope with the voices.

^a participant experienced a technical issue with the smartphone app.

^b participant dropped out due to conflicting study and work commitments during the trial.

Table 4

Summary statistics and effects on clinical measures for each group.

Note. Means and standard deviations incorporate pooled imputations; ***p < .001; **p < .01; *p < .05.

voices item remained significant ((F(1,30) = 27.03, p < .001), but was no longer significant for the VAS awareness of patterns in voices item (F(1,30) = 2.90, p = .09). Wilcoxon Signed Ranks tests indicated a marginally significant pre-post increase in the number of coping strategies (Z = -1.89, p = .06), but not the frequency of their use (Z = -0.37, p = .71), in the treatment group. There were no changes in the control group on the number of coping strategies (Z = -0.36, p = .72) nor frequency of their use (Z = -0.24, p = .81). Unplanned Pearson's correlations were run to examine whether confidence in using smartphone apps was related to any scores on

clinical or process measures within the treatment group at the outcome time point, with none reaching significance (p > .05). Sensitivity analyses were conducted which revealed minimal differences in the above analyses between imputed and non-imputed data sets (Thabane et al., 2013).

Fig. 3 displays box plots of PSYRATS-AH change scores in each group. There was an outlier in the treatment group representing a pre-post increase (i.e. worsening) in PSYRATS-AH total score. A review of assessment recordings and a follow-up qualitative interview indicated this was likely to be due to an external life event occurring around the time of the outcome assessment. A sensitivity analysis excluding this individual resulted in the effect on PSYRATS-AH becoming significant [(F(1,30) = 6.36, p < .05; Hedges g = 0.61, 95% CI = -0.07,1.33], with other results remaining the same.

3.5. Adverse events

Two events classed as serious adverse events (hospital admissions) according to the Australian National Health & Medical Research Council ([NHMRC], 2007) were reported—one from the treatment group and other from control. A review concluded these were unrelated to the trial or intervention.

4. Discussion

The findings of this study support the feasibility and acceptability of a brief coping-focused intervention for distressing voicehearing experiences which blended standard face-to-face psychological therapy with EMA/I between session. Completion rates of the EMA questionnaires were high, leading to the production of EMA-derived feedback in all attempted cases, and there was good engagement with both prompted and user-initiated EMI coping reminders. Despite minor technological issues, feedback regarding different aspects of the intervention was largely positive. These findings extend digital mental health research in psychosis by demonstrating that smartphone technologies can support standard face-to-face therapies; otherwise known as blended therapy (Erbe et al., 2017).

Research has demonstrated that EMI can support independent self-management of psychosis (Bell et al., 2017; Bucci et al., 2018; Schlosser et al., 2018). Our findings suggest that personalised EMI reminders of tailored self-management strategies determined during therapy may support the generalisation of these strategies into daily life. Participants were engaged with these EMI reminders and feedback suggested they were helpful. It is conceivable that personalised EMI reminders may be a simple and useful technology for other psychological treatment approaches. Future developments may involve more streamlined programming of EMI content and the use of context-aware systems to determine the timing and nature of tailored EMI prompts (Bakker et al., 2016; Burns et al., 2011; Price et al., 2014; Proudfoot, 2013).

A significant novelty of this study is the analysis of within-person EMA data to inform clinical formulation in psychological therapy. No other study has examined this application of EMA in psychosis, despite considerable interest (Ebner-Priemer and Trull, 2009; Firth and Torous, 2015; McDevitt-Murphy et al., 2018; Myin-Germeys et al., 2016; Oorschot et al., 2012; Trull and Ebner-Priemer, 2009). Although one case experienced a technological issue whereby the alerts were not consistently received, the overall high completion rate of the EMA questionnaires led to the production of EMA-derived feedback in all attempted cases. This, alongside predominantly positive feedback from participants, supports the feasibility and acceptability of this approach. The high level of engagement with both EMA and EMI components are consistent with other similar studies (Ben-Zeev et al., 2014; Berkel et al., 2017; Bucci et al., 2018; Firth and Torous, 2015; Kumar et al., 2018), although recent findings suggest naturalistic engagement with apps may be lower (Torous et al., 2017). Notably, one participant who dropped out commented on the effort involved in the intervention, suggesting this approach may be more difficult for those with limited time. Whilst we did not find evidence of a relationship between the characteristics of the sample and engagement with, nor effects of, the intervention, further research in this area would be beneficial to identify what works best and for whom (Michie et al., 2017; Ritterband et al., 2006).

It is hoped that this study, being the first of its kind, spurs further research exploring different statistical and methodological approaches to conducting within-person analysis of EMA data for clinical purposes. We used regression analyses and summary statistics, however other statistical approaches, such as network analysis (Bringmann et al., 2013), machine learning (Burns et al.,

• 2011) or dynamic factor modelling (Fisher, 2015), may also be appropriate (Barnett et al., 2018; Myin-Germeys et al., 2016). Automated analyses which produce simple, accurate, meaningful and engaging data visualisations, possibly in real time, would also be beneficial. Further research should also explore the most appropriate EMA sampling schedule for different clinical purposes and the development of validated EMA scales (Firth and Torous,

• 2015).

Although only a pilot trial, post-intervention effects were in a direction favouring the treatment group for the primary clinical outcome of overall severity of voices, and to a lesser extent the secondary outcome of negative impact of the voices, but not emotional distress. More proximal, process, measures indicated statistically significant improvements favouring the treatment group in confidence in coping with voices, awareness of factors influencing voices, and close to significant increase in the number (but not frequency) of coping strategies used. These findings suggest this intervention holds promise for reducing the overall severity of voices and their negative impact, possibly occurring via the process of improved coping and understanding of voices. The very large effects observed on the measure of confidence in coping with voices provides proof-of-concept evidence for the mechanisms of the intervention. As these processes were targeted directly by EMA/I, this suggests the technology component was of benefit. These effects appear consistent with prior trials of CSE-based interventions (Hayward et al., 2018; Paulik et al., 2018; Tarrier et al., 1993, 1998; Yusupoff and Tarrier, 1996), however direct comparisons are limited due to variations in methodology. Notably, one participant within the treatment group showed a worsening in the primary outcome measure which may have lowered our conservative estimate of the average effect size. Whilst our investigation suggested an external life event was the main contributing factor,

8

this highlights caution in ensuring adverse events are considered in any larger scale trialling, an area possibly neglected in CBT for psychosis research (Morrison, 2018).

The following limitations should be recognised. Firstly, it is unclear if the effects of the intervention were maintained as there was no follow-up time-point. Secondly, the intervention contained multiple components, limiting inferences regarding specific therapeutic mechanisms. A future trial should carefully consider an active comparison group (e.g., Bucci et al., 2018), including the recently highlighted digital placebo effect (Torous and Firth, 2016). Dismantling studies to isolate the active ingredients of digital interventions may assist in refining the features which yield maximum benefits, whilst improving our understanding of the mechanisms which drive them (Collins et al., 2007; Michie et al., 2017). Thirdly, the small sample size limits generalisability and clearly a fully powered trial is needed to determine clinical efficacy. Fourth, group difference in negative symptoms at baseline were observed, however it is noted that subsequent analyses controlling for this variable resulted in only minor changes.

Conclusion

The current study demonstrates the clinical potential of EMA/I as tools within blended therapies for psychotic experiences. This justifies potential further development of a purpose-built mobile app with evaluation in a full scale RCT to determine efficacy, and potentially investigations within other psychological treatment approaches and clinical populations.

Declaration of competing interest

The authors declare there are no conflicts of interest.

Acknowledgments

The authors thank the participants, lived experience consultants, and researchers Professor Denny Meyer, Dr Wei Lin Toh, Dr Rachel Brand, Ms Inge Gnatt and Ms Louise Moncur.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.schres.2019.10.026.

Contributors

IHB and NT conceptualised the intervention. IHB led the design of the protocol, with contributions from all other authors. IHB conducted the trial, with oversight by NT. SLR and NT assisted with the analysis, which was conducted by IHB. IHB wrote the first draft of this manuscript, with contributions from all other authors. All authors contributed to and have approved the final manuscript.

Funding

This research was supported by the Australian Government Research Training Program Scholarship and the Barbara Dicker Brain Sciences Foundation Grant Scheme.

References

Alvarez-Jimenez, M., Alcazar-Corcoles, M., Gonzalez-Blanch, C., Bendall, S., McGorry, P., Gleeson, J., 2014. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr. Res. 156 (1), 96-106.

Andreasen, N.C., 1989. Scale for the assessment of negative symptoms (SANS). Br. J. Psychiatry 155 (Suppl 7), 53-58.

Bakker, D., Kazantzis, N., Rickwood, D., Rickard, N., 2016. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Mental Health 3 (1).

Barnett, I., Torous, J., Staples, P., Keshavan, M., Onnela, J.P., 2018. Beyond smartphones and sensors: choosing appropriate statistical methods for the analysis of longitudinal data. J. Am. Med. Inform. Assoc. 25 (12), 1669-1674.

Bell, I.H., Lim, M.H., Rossell, S.L., Thomas, N., 2017. Ecological momentary assessment and intervention in the treatment of psychotic disorders: a systematic review. Psychiatr. Serv. 68 (11), 1172-1181.

Bell, I.H., Fielding-Smith, S.F., Hayward, M., Rossell, S.L., Lim, M.H., Farhall, J., Thomas, N., 2018a. Smartphone-based ecological momentary assessment and intervention in a blended coping-focused therapy for distressing voices: development and case illustration. Internet Interventions 14, 18-25. https:// doi.org/10.1016/j.invent.2018.11.001.

Bell, I.H., Fielding-Smith, S.F., Hayward, M., Rossell, S.L., Lim, M.H., Farhall, J., Thomas, N., 2018b. Smartphone-based ecological momentary assessment and intervention in a coping-focused intervention for hearing voices (SAVVy): study protocol for a pilot randomised controlled trial. Trials 19 (1), 262. https:// doi.org/10.1186/s13063-018-2607-6.

Ben-Zeev, D., Brenner, C.J., Begale, M., Duffecy, J., Mohr, D.C., Mueser, K.T., 2014. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr. Bull. 40 (6), 1244-1253.

Berkel, N.V., Ferreira, D., Kostakos, V., 2017. The experience sampling method on mobile devices. ACM Comput. Surv. 50 (6), 93.

Berry, K., Haddock, G., 2008. The implementation of the NICE guidelines for schizophrenia: barriers to the implementation of psychological interventions and recommendations for the future. Psychol. Psychother. Theory Res. Pract. 81 (4), 419-436.

Brenner, C., Ben-Zeev, D., 2014. Affective forecasting in schizophrenia: comparing predictions to real-time Ecological Momentary Assessment (EMA) ratings. Psychiatr. Rehabil. J. 37 (4), 316.

Bringmann, L.F., Vissers, N., Wichers, M., Geschwind, N., Kuppens, P., Peeters, F., et al., 2013. A network approach to psychopathology: new insights into clinical longitudinal data. PLoS One 8 (4), e60188.

Bucci, S., Barrowclough, C., Ainsworth, J., Machin, M., Morris, R., Berry, K., et al., 2018a. Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophrenia Bulletin, p. sby032.

Bucci, S., Morris, R., Berry, K., Berry, N., Haddock, G., Barrowclough, C., et al., 2018b. Early psychosis service user views on digital technology: qualitative analysis. JMIR Mental Health 5 (4), e10091.

Burns, M.N., Begale, M., Duffecy, J., Gergle, D., Karr, C.J., Giangrande, E., Mohr, D.C., 2011. Harnessing context sensing to develop a mobile intervention for depression. JMIR 13 (3), e55.

Collins, L.M., Murphy, S.A., Strecher, V., 2007. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am. J. Prev. Med. 32 (5), S112-S118.

Delespaul, P., 1995. Assessing Schizophrenia in Daily Life - the Experience Sampling Method. Maastricht. IPSER Foundation, Netherlands.

Devilly, G.J., Borkovec, T.D., 2000. Psychometric properties of the credibility/ex-pectancy questionnaire. J. Behav. Ther. Exp. Psychiatry 31 (2), 73-86.

Dixon, L.B., Holoshitz, Y., Nossel, I., 2016. Treatment engagement of individuals experiencing mental illness: review and update. World Psychiatry 15 (1), 13-20.

Durlak, J.A., 2009. How to select, calculate, and interpret effect sizes. J. Pediatr. Psychol. 34 (9), 917-928.

Ebner-Priemer, U.W., Trull, T.J., 2009. Ambulatory assessment: an innovative and promising approach for clinical psychology. Eur. Psychol. 14 (2), 109-119.

Enders, C.K., 2001. A primer on maximum likelihood algorithms available for use with missing data. Struct. Equ. Model. 8 (1), 128-141.

Erbe, D., Eichert, H.-C., Riper, H., Ebert, D.D., 2017. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: systematic review. JMIR 19 (9).

Farhall, J., Thomas, N., 2013. Cognitive and behavioural therapies for psychosis. Aust. N. Z. J. Psychiatr. 47 (6), 508-511. https://doi.org/10.1177/0004867413483370.

Fioravanti, M., Bianchi, V., Cinti, M.E., 2012. Cognitive deficits in schizophrenia: an updated metanalysis of the scientific evidence. BMC Psychiatry 12 (1), 64.

First, M.B., Williams, J.B.W., Karg, R.S., Spitzer, R.L., 2015. Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). American Psychiatric Association, Arlington, VA.

Firth, J., Torous, J., 2015. Smartphone apps for schizophrenia: a systematic review. JMIR mHealth and uHealth 3 (4), e102.

Firth, J., Cotter, J., Torous, J., Bucci, S., Firth, J.A., Yung, A.R., 2015. Mobile phone ownership and endorsement of “mHealth” among people with psychosis: a meta-analysis of cross-sectional studies. Schizophr. Bull. 42 (2), 448-455.

Fisher, A.J., 2015. Toward a dynamic model of psychological assessment: implications for personalized care. J. Consult. Clin. Psychol. 83 (4), 825.

Galletly, C., Castle, D., Dark, F., Humberstone, V., Jablensky, A., Killackey, E., et al., 2016. Royal Australian and New Zealand College of Psychiatrists clinical practice guidelines for the management of schizophrenia and related disorders. Aust. N. Z. J. Psychiatr. 50 (5), 410-472.

Gaudiano, B.A., Busch, A.M., Wenze, S.J., Nowlan, K., Epstein-Lubow, G., Miller, I.W., 2015. Acceptance-based behavior therapy for depression with psychosis: results from a pilot feasibility randomized controlled trial. J. Psychiatr. Pract. 21 (5), 320.

Gay, K., Torous, J., Joseph, A., Pandya, A., Duckworth, K., 2016. Digital technology use

among individuals with schizophrenia: results of an online survey. JMIR Mental Health 3 (2).

Granholm, E., Loh, C., Swendsen, J., 2008. Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophr. Bull. 34 (3), 507-514.

Haddock, G., McCarron, J., Tarrier, N., Faragher, E., 1999. Scales to measure dimensions of hallucinations and delusions: the psychotic symptom rating scales (PSYRATS). Psychol. Med. 29 (04), 879-889.

Haddock, G., Wood, L., Watts, R., Dunn, G., Morrison, A.P., Price, J., 2011. The Subjective Experiences of Psychosis Scale (SEPS): psychometric evaluation of a scale to assess outcome in psychosis. Schizophr. Res. 133 (1), 244-249.

Haddock, G., Eisner, E., Boone, C., Davies, G., Coogan, C., Barrowclough, C., 2014. An investigation of the implementation of NICE-recommended CBT interventions for people with schizophrenia. J. Ment. Health 23 (4), 162-165.

Hatcher, R.L., Gillaspy, J.A., 2006. Development and validation of a revised short version of the Working Alliance Inventory. Psychother. Res. 16 (1), 12-25.

Hayward, M., Edgecumbe, R., Jones, A.M., Berry, C., Strauss, C., 2018. Brief coping strategy enhancement for distressing voices: an evaluation in routine clinical practice. Behav. Cognit. Psychother. 46 (2), 226-237.

Hazell, C.M., Hayward, M., Cavanagh, K., Jones, A.M., Strauss, C., 2018. Guided selfhelp cognitive-behaviour Intervention for VoicEs (GiVE): results from a pilot randomised controlled trial in a transdiagnostic sample. Schizophr. Res. 195, 441-447.

Heron, K., Smyth, J., 2010. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br. J. Health Psychol. 15 (1), 1-39.

Ince, P., Haddock, G., Tai, S., 2016. A systematic review of the implementation of recommended psychological interventions for schizophrenia: rates, barriers, and improvement strategies. Psychol. Psychother. Theory Res. Pract. 89 (3), 324-350.

Jauhar, S., McKenna, P., Radua, J., Fung, E., Salvador, R., Laws, K., 2014. Cognitive-behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. Br. J. Psychiatry 204 (1), 20-29.

Johansen, R., Hestad, K., Iversen, V.C., Agartz, I., Sundet, K., Andreassen, O.A., Melle, I., 2011. Cognitive and clinical factors are associated with service engagement in early-phase schizophrenia spectrum disorders. J. Nerv. Ment. Dis. 199 (3), 176-182.

Julious, S.A., 2005. Sample size of 12 per group rule of thumb for a pilot study. Pharm. Stat.: The Journal ofApplied Statistics in the Pharmaceutical Industry 4 (4), 287-291.

Kumar, D., Tully, L.M., Iosif, A.-M., Zakskorn, L.N., Nye, K.E., Zia, A., Niendam, T.A., 2018. A mobile health platform for clinical monitoring in early psychosis: implementation in community-based outpatient early psychosis care. JMIR Mental Health 5 (1).

Lovibond, P.F., Lovibond, S.H., 1995. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories. Behav. Res. Ther. 33 (3), 335-343.

McDevitt-Murphy, M.E., Luciano, M.T., Zakarian, R.J., 2018. Use of ecological momentary assessment and intervention in treatment with adults. Focus 16 (4), 370-375.

National Health and Medical Research Council, 2007. National statement on ethical conduct in human research 2007 (Updated May 2015). Retrieved from. https:// nhmrc.gov.au/about-us/publications/national-statement-ethical-conduct-human-research-2007.

Michie, S., Yardley, L., West, R., Patrick, K., Greaves, F., 2017. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. JMIR 19 (6).

Morrison, A.P., 2018. Should people with psychosis be supported in choosing cognitive therapy as an alternative to antipsychotic medication: a commentary on current evidence. Schizophrenia Research.

MovisensXS. Retrieved from. https://xs.movisens.com/.

Myin-Germeys, I., Klippel, A., Steinhart, H., Reininghaus, U., 2016. Ecological momentary interventions in psychiatry. Curr. Opin. Psychiatr. 29 (4), 258-263.

National Institute for Health and Care Excellence [NICE], 2010. Core Interventions in the Treatment and Mangement of Schizophrenia in Adults in Primary and Secondary Care, updated edition. UK The British Psychological Society and The Royal College of Psychiatrists, London.

Oorschot, M., Lataster, T., Thewissen, V., Wichers, M., Myin-Germeys, I., 2012. Mobile assessment in schizophrenia: a data-driven momentary approach. Schiz-ophr. Bull. 38 (3), 405-413.

Palmier-Claus, J.E., Ainsworth, J., Machin, M., Barrowclough, C., Dunn, G., Barkus, E., et al., 2012. The feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry 12 (1), 172.

Palmier-Claus, J., Myin-Germeys, I., Barkus, E., Bentley, L., Udachina, A., Delespaul, P., et al., 2011. Experience sampling research in individuals with mental illness: reflections and guidance. Acta Psychiatr. Scand. 123 (1), 12-20.

Paulik, G., Jones, A.M., Hayward, M., 2018. Brief coping strategy enhancement for distressing voices: P redictors of engagement and outcome in routine clinical practice. Clin. Psychol. Psychother. 25 (5), 634-640.

Price, M., Yuen, E.K., Goetter, E.M., Herbert, J.D., Forman, E.M., Acierno, R., Ruggiero, K.J., 2014. mHealth: a mechanism to deliver more accessible, more effective mental health care. Clin. Psychol. Psychother. 21 (5), 427-436.

Proudfoot, J., 2013. The future is in our hands: the role of mobile phones in the

9 prevention and management of mental disorders. Aust. N. Z. J. Psychiatr. 47 (2), 111-113. https://doi.org/10.1177/0004867412471441.

QMinim. Retrieved from http://qminim.sourceforge.net.

Ritterband, L.M., Andersson, G., Christensen, H.M., Carlbring, P., Cuijpers, P., 2006. Directions for the international society for research on internet interventions (ISRII). JMIR 8 (3).

Schizophrenia Commission, T.S., 2012. The Abandoned Illness: a Report from the Schizophrenia Commission. Illness RM, London.

Schlosser, D.A., Campellone, T.R., Truong, B., Etter, K., Vergani, S., Komaiko, K., Vinogradov, S., 2018. Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophr. Bull. 44 (5), 1010-1020.

Sheehan, D.V., Lecrubier, Y., Sheehan, K.H., Amorim, P., Janavs, J., Weiller, E., et al., 1998. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.J. Clin. Psychiatry 59 (Suppl 20), 22-33.

Shiffman, S., Stone, A., Hufford, M., 2008. Ecological momentary assessment. Annu. Rev. Clin. Psychol. 4, 1-32.

Sim, J., Lewis, M., 2012. The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. J. Clin. Epidemiol. 65 (3), 301-308.

SPSS Inc, 2017. IBM SPSS Statistics for Windows (Version. IBM Corp, Armonk, NY, version 25.0.

StataCorp, 2015. Stata Statistical Software: Release 14. StataCorp LP, College Station, TX.

Stone, A., Shiffman, S., Atienza, A., Nebeling, L., 2007. The Science of Real-Time Data Capture: Self-Reports in Health Research. Oxford University Press.

Tabachnick, B.G., Fidell, L.S., 2007. Using Multivariate Statistics, fifth ed. Allyn & Bacon/Pearson Education, Boston, USA.

Tarrier, N., 1992. Management and modification of residual positive psychotic symptoms. In: Birchwood, M., Tarrier, N. (Eds.), Innovations in the Psychological Management of Schizophrenia: Assessment, Treatment and Services. Wiley Series in Clinical Psychology, pp. 147-169.

Tarrier, N., Harwood, S., Yusopoff, L., Beckett, R., Baker, A., 1990. Coping strategy enhancement (CSE): a method of treating residual schizophrenic symptoms. Behav. Psychother. 18 (04), 283-293.

Tarrier, N., Beckett, R., Harwood, S., Baker, A., Yusupoff, L., Ugarteburu, I., 1993. A trial of two cognitive-behavioural methods of treating drug-resistant residual psychotic symptoms in schizophrenic patients: I. Outcome. Br. J. Psychiatry 162 (4), 524-532.

Tarrier, N., Yusupoff, L., Kinney, C., McCarthy, E., Gledhill, A., Haddock, G., Morris, J., 1998. Randomised controlled trial of intensive cognitive behaviour therapy for patients with chronic schizophrenia. BMJ 317 (7154), 303-307.

Teare, M.D., Dimairo, M., Shephard, N., Hayman, A., Whitehead, A., Walters, S.J., 2014. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials 15 (1), 1.

Thabane, L., Mbuagbaw, L., Zhang, S., Samaan, Z., Marcucci, M., Ye, C., et al., 2013. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Med. Res. Methodol. 13 (1), 92.

Thomas, N., Bless, J.J., Alderson-Day, B., Bell, I.H., Cella, M., Craig, T., et al., 2019. Potential applications of digital technology in assessment, treatment, and selfhelp for hallucinations. Schizophr. Bull. 45 (Suppl. ment_1), S32-S42.

Torous, J., Firth, J., 2016. The digital placebo effect: mobile mental health meets clinical psychiatry. The Lancet Psychiatry 3 (2), 100-102.

Torous, J., Friedman, R., Keshavan, M., 2014. Smartphone ownership and interest in mobile applications to monitor symptoms of mental health conditions. JMIR mHealth and uHealth 2 (1), e2.

Torous, J., Staples, P., Slaters, L., Adams, J., Sandoval, L., Onnela, J., Keshavan, M., 2017. Characterizing smartphone engagement for schizophrenia: results of a naturalist mobile health study. Clin. Schizophrenia Relat. Psychoses. In-Press. [Epub ahead of print].

Treisman, G.J., Jayaram, G., Margolis, R.L., Pearlson, G.D., Schmidt, C.W., Mihelish, G.L., et al., 2016. Perspectives on the use of eHealth in the management of patients with schizophrenia. J. Nerv. Ment. Dis. 204 (8), 620.

Trull, T., Ebner-Priemer, U., 2009. Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol. Assess. 21 (4), 457-462.

Turkington, D., Kingdon, D., Weiden, P.J., 2008. Cognitive behavior therapy for schizophrenia. Am. J. Psychiatry 6 (2), 257-266.

van der Gaag, M., Valmaggia, L.R., Smit, F., 2014. The effects of individually tailored formulation-based cognitive behavioural therapy in auditory hallucinations and delusions: a meta-analysis. Schizophr. Res. 156 (1), 30-37.

Webb, C.A., Kertz, S.J., Bigda-Peyton, J.S., Bjorgvinsson, T., 2013. The role of pretreatment outcome expectancies and cognitive-behavioral skills in symptom improvement in an acute psychiatric setting. J. Affect. Disord. 149 (1-3), 375-382.

Wechsler, D., 2001. Wechsler Test of Adult Reading (WTAR). The Psychological Corporation, San Antonio, TX.

White, R., Gumley, A., McTaggart, J., Rattrie, L., McConville, D., Cleare, S., Mitchell, G., 2011. A feasibility study of Acceptance and Commitment Therapy for emotional dysfunction following psychosis. Behav. Res. Ther. 49 (12), 901-907.

Yusupoff, L., Tarrier, N., 1996. Coping strategy enhancement for persistent hallucinations and delusions. Cognitive-Behavioural Interventions with Psychotic Disorders 86-102.

Schizophrenia Bulletin Advance Access published March 8, 2014

Feasibility, Acceptability, and Preliminary Efficacy of a Smartphone Intervention for Schizophrenia

Dror Ben-Zeev*,¹, Christopher J. Brenner², Mark Begale³, Jennifer Duffecy³, David C. Mohr³, and Kim T. Mueser^1,4

'Department of Psychiatry, Dartmouth Psychiatric Research Center, Geisel School of Medicine at Dartmouth, Lebanon, NH; ²Thresholds, Chicago, IL; ³Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, IL; ⁴Center for Psychiatric Rehabilitation, Sargent College of Health and Rehabilitation Sciences, Boston University, Boston, MA

*To whom correspondence should be addressed; Dartmouth Psychiatric Research Center, 85 Mechanic Road, Lebanon, NH 03766, US; tel: 603-448-0263, fax: 603-448-3976, e-mail: dror.ben-zeev@dartmouth.edu

The FOCUS smartphone intervention was developed to provide automated real-time/real-place illness management support to individuals with schizophrenia. The system was specifically designed to be usable by people with psychotic disorders who may have cognitive impairment, psychotic symptoms, negative symptoms, and/or low reading levels. FOCUS offers users both prescheduled and on-demand resources to facilitate symptom management, mood regulation, medication adherence, social functioning, and improved sleep. In this study, 33 individuals with schizophrenia or schizoaffective disorder used FOCUS over a 1month period in their own environments. Participants were able to learn how to use the intervention independently, and all but one participant completed the trial successfully and returned the smartphones intact. Completers used the system on 86.5% of days they had the device, an average of 5.2 times a day. Approximately 62% of use of the FOCUS intervention was initiated by the participants, and 38% of use was in response to automated prompts. Baseline levels of cognitive functioning, negative symptoms, persecutory ideation, and reading level were not related to participants’ use of the intervention. Approximately 90% of participants rated the intervention as highly acceptable and usable. Paired samples t tests found significant reductions in psychotic symptoms, depression, and general psychopathology, after 1 month of FOCUS use. This study demonstrated the feasibility, acceptability, and preliminary efficacy of the FOCUS intervention for schizophrenia and introduces a new treatment model which has promise for extending the reach of evidence-based care beyond the confines of a physical clinic using widely available technologies.

Key words: Mobile Health (mHealth)/mobile interventions/depression/hallucinations/medication adherence/sleep/social functioning

Introduction

Schizophrenia is associated with high costs to individuals, their families, and society.¹,² Poorly managed, the illness can cause significant personal distress and impairment and is associated with increased risk for depression, anxiety, substance use, homelessness, victimization, hospitalization, and suicide.^3-8

Over the last 2 decades several evidence-based psychosocial interventions have been developed to help individuals with schizophrenia better cope with symptoms, improve social functioning, maintain a healthier lifestyle, and engage in meaningful work, even in the context of a chronic mental health condition.^9,10 However, these interventions are rarely available at clinical settings for a variety of reasons, including the lack of clinicians who are trained in these approaches, limited funding for psychosocial interventions, and poor utilization and ongoing engagement in treatments even when they are available.^11-13

Mobile Health (mHealth) approaches that leverage mobile devices such as cellular phones and smartphones to support healthcare are promising for deployment of interventions that are unconstrained by the limitations of existing treatment settings.^14,15 Mobile phones are carried on the person, typically turned on, and have near constant connectivity and access to multimedia resources. Thus, they can serve as conduits for interventions any time, and in almost any location.¹⁶ Furthermore, mobile phones are widely available, affordable, and are continuously dropping in cost; there are now over 6 billion mobile phones subscriptions worldwide, with the majority being used in low and middle income countries.¹⁷ In the United States, underserved populations now use “smartphones” (ie, mobile phones with computational capacities) as their primary method for accessing resources on the internet,¹⁸ and there is evidence that even homeless individuals with limited resources own and use mobile phones regularly.¹⁹-²⁰ Mobile phone use among people with severe psychiatric disabilities is not dramatically different from that observed in the general population—a recent survey conducted among 1592 adults with serious mental illnesses found that 72% of respondents had mobile phones and used them for a range of functions including calling, texting, and email. Moreover, many individuals indicated that they would be interested in engaging in mobile interventions (eg, reminders, psychoeducation, contact with clinicians) via their mobile device.²¹

Early efforts to incorporate mobile phones into the clinical care of people with schizophrenia have produced mixed results. In one pilot study, participants received daily text message assessments sent to basic mobile phones from a remote server.²² If participants responded, the server engaged them with follow-up suggestions to improve their symptoms and functioning. Participants responded to 86% of assessments over 3 months, but clinical outcomes did not significantly change from baseline to posttrial. Some changes in self-reported distress from auditory hallucinations, social functioning, and medication adherence were recorded, but participants with lower cognitive functioning and more severe negative symptoms had greater difficulty negotiating the requirements of the protocol. The investigative team concluded that there was a need to develop interventions that capitalize on emerging smartphone technology that could enable more userfriendly and potent interventions.

In a second study, high risk for relapse outpatients and their family members received weekly text message requests to complete assessments of early warning signs of relapse on their mobile phones as part of a randomized controlled trial of an information technology-aided relapse prevention program.²³ When participants reported problems, automated alerts were sent to their treating psychiatrists with a prompt to follow-up with a medication evaluation. Participants responded to 80% of weekly requests over a year, but practitioner adherence to the protocol was low and in many cases follow-up steps were not taken. When clinicians did respond accordingly, individuals in the intervention arm did significantly better than controls in terms of hospitalization, number of inpatient days, and costs. Taken together, these studies demonstrate that people with schizophrenia can successfully engage in mobile interventions, but that these should be user-centered in terms of the technology (ie, user-friendly, intuitive, engaging) and treatment model (ie, include self-management components that are independent of clinician engagement).

FOCUS: A User-Centered Smartphone System for Schizophrenia

Working together with patients and clinicians at community settings, our research team developed FOCUS, a smartphone system designed to support self-management of illness in individuals with schizophrenia. The system is grounded in 2 theoretical models: the cognitive model of psychosis^24,25 and the stress-vulnerability model of schizophrenia.^26,27 FOCUS aims to identify and dismantle dysfunctional beliefs that contribute to maintenance and distress associated with symptoms, and to interrupt the cyclical relationship between stress (eg, fatigue, interpersonal conflict, social isolation, poor medication adherence) and vulnerability that may lead to illness exacerbation.²⁸ Through several iterative cycles of development and user feedback, we constructed a smartphone system that targets symptoms of psychosis, social functioning, mood problems, medication adherence, and sleep difficulties. We drew treatment content from an array of evidence-based interventions (ie, cognitive restructuring, behavioral tailoring, social skills training, illness management and recovery, anger management, behavioral activation, sleep hygiene), and adapted it so that it was suitable for delivery via smartphones. Content was distilled into brief interactive exchanges that are accompanied by illustrative images (ie, photographs, cartoons, touchscreen reminder buttons) that are displayed on the smartphone screen. To maximize usability, the system was developed in accordance with design principles for electronic resources for people with serious mental illness and cognitive impairment.²⁹

FOCUS was initially tested with individuals with schizophrenia in laboratory conditions, and the system was adapted based on our observations of problems and user recommendations.³⁰ Once we were satisfied that the system functioned well in controlled environments, we conducted a field trial in which 33 participants were provided with a smartphone installed with the FOCUS system, to use over 1 month in their own environments. We hypothesized that participants would find the system acceptable, usable, engaging, and helpful. We also hypothesized that participants’ cognitive functioning, negative symptoms, and levels of persecutory ideation would not impact their use of the system. To our knowledge, this is the first deployment of a smartphone intervention for schizophrenia.

Methods

Participants

The study was approved by the Committee for Protection of Human Subjects at Dartmouth. Thirty-three individuals with schizophrenia or schizoaffective disorder were recruited from community-based treatment programs in Chicago. Participants had a mean age of 45.9 (SD = 8.78). The sample was 61% male, 76% African American, 21% white, and 3% more than one race. Six percent identified as Latino. Participants reported an average of 12.7 years of education (SD = 2.32), but their average reading level was at the eighth grade level (assessed using the reading subsection in Wide Range Achievement Test—Fourth Edition, Wilkinson and Robertson³¹ [WRAT-4]). Sixty-one percent were living independently, 21% resided in a supervised living facility, and 18% were living with family members. The majority were unemployed (87.9%) and owned a mobile phone of some kind (87.5%). Of mobile phone users, 32% owned a smartphone. Participants reported an average of 6.40 (SD = 4.21) lifetime psychiatric hospitalizations. At baseline, the sample experienced moderate symptoms of schizophrenia (Positive and Negative Syndrome Scale, Kay et al³² [PANSS] total, M = 77.64, SD = 4.23; PANSS-positive score, M = 19.24, SD = 4.23; PANSS-negative score, M = 17.76, SD = 3.60), mild depressive symptoms (Beck Depression Inventory-Second Edition, Beck et al³³ [BDI-2], M = 19.52, SD = 8.86), and subthreshold clinical insomnia (Insomnia Severity Index, Morin et al³⁴ [ISI], M = 12.27, SD = 6.24). Participants’ beliefs that it was necessary for them to take medications were stronger than their medication-related concerns (Brief Medication Questionnaire, Svarstad et al³⁵ [BMQ] necessity-concern differential, M = 5.15, SD = 5.69). On average, participants had moderate cognitive impairment (Brief Assessment of Cognition in Schizophrenia³⁶ [BACS] t-score, M = 30.09, SD = 11.46).

Procedures

Participant Screening. Clinical staff identified 95 individuals who were viable candidates for the study (ie, chart diagnosis, willingness to being contacted for research projects). Research staff contacted this group, and 58 expressed interest in participating in the study. After providing written informed consent, all potential participants were administered a structured diagnostic interview (Structured Clinical Interview for DSM-IV Axis I Disorders-Patient Edition, First et al³⁷) to verify diagnosis of schizophrenia or schizoaffective disorders. Potential participants then completed a battery of laboratory-based self-report and interview measures that included demographic information, measures of symptoms of schizophrenia (PANSS), symptoms of depression (BDI-2), and sleep difficulties (ISI). Potential participants were enrolled if they were 18 years of age or older, prescribed psychotropic medications, and had “mild” or higher severity scores on 2 of the following: hallucinatory behavior on the PANSS (P3 > 3), passive/apathetic social avoidance or active social avoidance on the PANSS (N4 or G16 > 3), BDI-2 total score > 10, or ISI score > 15. Candidates were excluded if they had hearing, vision, or motor impairments that made it impossible for them to use a smartphone (assessed on-site by study staff), if their reading level was below fourth grade, or if they were enrolled in another intervention study. Five individuals were found to be ineligible due to diagnosis, 6 were ineligible due to reading level, 13 were ineligible due to symptom severity criteria, and 1 was excluded due to enrollment in another study.

Pretrial. Once enrolled, participants returned within a week and were administered the BACS and BMQ. This visit also included a shared decision-making session with research staff to establish the treatment targets each participant would work on (ie, receive daily prompts and content from the FOCUS system) over the course of the field trial. Every participant was assigned medication adherence as a treatment target because we wanted to have at least one element that was consistent across all participants, and this was identified as a high priority area by clinical staff. Two additional domains were chosen from the following options: social (encompasses interventions targeting persecutory ideation, anger management, and social skills training), mood problems (ie, depression and anxiety), auditory hallucinations, or sleep difficulties. The domain selection was informed by participants’ preference and data from the screening and baseline assessments; a study interviewer would identify measure scores that were particularly high and record concrete examples participants provided when asked about areas of greatest concern (eg, avoiding locations where voices were loudest, panic attacks, staying up at night ruminating). The interviewer would then reflect back to the participant their assessment of 2 “high priority” areas, and inquire whether they agree and would like to focus on these targets or if they preferred a different combination. Once they decided on the domains, the interviewer and participant would review their “typical day” and discuss when it would make most sense to address each domain (ie, the times they would get daily FOCUS prompts to engage).

Following treatment target selection, a 30-min training session was administered focusing on use of the smartphone provided and the different elements of the FOCUS system. Training on the use of the device included how to charge the phone, turning the phone on and off as well as locking the screen, how to use a touchscreen, how to make and receive calls, and how to save contact numbers. The FOCUS system was then demonstrated by a trained research assistant. The demonstration focused on in-the-moment use and selection of resources from the different on-demand options. Participants then had the opportunity to practice using FOCUS and ask questions as needed. Individuals were given a Motorola Droid 4 smartphone running the Android 4.1 operating system and charger and commenced their participation in the field trial only after they demonstrated proficiency in using the device and the FOCUS system in the laboratory. They were instructed to charge the phone at night and carry it with them wherever they go, but otherwise to go about their daily life as usual.

Field Trial. Over the course of 1 month, the FOCUS system prompted participants to complete an assessment 3 times daily, on one of each of the 3 treatment targets between the hours of 9 am and 1 pm, 1 and 5 pm, and 5 and 9 pm (exact times within those ranges were determined randomly daily by the system). Participants were asked to respond to each system-generated assessment whenever possible. In response to the content of participant entries, the FOCUS system deployed tailored in-the-moment interventions. Participants were also instructed to use the host of on-demand FOCUS features as often as they liked, whenever they needed support. The FOCUS system automatically uploaded participant use data (ie, response to prompt rate, content of their responses to assessment, self-initiated use of resources) to a secure study server. Thus, so long as the smartphone was within reception, the research team could view user response data continuously. Study staff called each participant once per week to check-in and assist with any technical difficulties.

Posttrial. After 1 month of use, participants returned the charger and smartphone and completed the PANSS, BDI-2, ISI, and BMQ, for a second time. Before debriefing, participants also completed a 26-item self-report acceptability/usability measure comprised of adapted items from the System Usability Scale,³⁸ Post Study System Usability Questionnaire,³⁹ Technology Assessment Model Measurement Scales,⁴⁰ and Usefulness, Satisfaction, and Ease questionnaire.⁴¹ Participants were asked to rate their agreement with a series of statements about the intervention (see table 1 for all items).

Participants were paid for the time they devoted to pre-and posttrial assessments. They received an unlimited data plan that enabled unrestricted calling, texting, and internet use on the smartphone during the trial. Daily engagement in the FOCUS intervention was not incentivized and participants were instructed to use the system as often as they wanted.

Description of the Mobile Intervention

The FOCUS system is comprised of 3 applications (apps) that are installed onto the smartphone, and a web-based dashboard. The first app prompts users to engage daily via auditory signals and visual notifications that appear on the screen. The second is the primary FOCUS app that uses interactive algorithms to generate brief assessments and interventions that the user progresses through using touchscreen buttons on the smartphone homescreen. The third is a Quick Tips app that allows users to access illness self-management resources and suggested coping strategies from a menu of options.

Once signaled by the prompting app (see figure 1), users can decide to engage or ignore the prompt. If they engage, the system will launch a brief assessment of their current status (eg, “How has your mood been today?”) with multiple choice touchscreen response options that appear below the question on the same screen. If the user endorses difficulties (eg, “Very bad. I’m very upset”) the system provides feedback (eg, “Looks like you could use some support. FOCUS is happy to help.”), followed by a more in-depth assessment (“Have you had any of these thoughts lately?”). Users’ responses determine the nature of the subsequent interventions they will receive (eg, see figure 2). Once they complete a sequence of screens, making selections as they progress and receiving interventions, the system signs off (“Thank you for using FOCUS. Have a nice day.”) until the next scheduled prompt.

Users can access all intervention content “on-demand” whenever and wherever they choose, by going to the FOCUS homescreen and selecting any of the 5 treatment target icons (see figure 1) or by accessing Quick Tips for briefer noninteractive content. Each intervention sequence has multiple wording and image variations so that users do not encounter the exact same intervention every time, even if they make similar selections (see examples for FOCUS intervention screens for each target area in figure 2). When the smartphone has wireless connectivity, the FOCUS application transmits the use data to a secure study server. The data is then displayed as a continuously updated report on a secure web-based dashboard that the research team or authorized clinicians can access at any time.

Overview of Analyses

Descriptive statistics were derived from participants’ smartphone use data to characterize feasibility and acceptability. Pearson product-moment correlation coefficients were used to examine the association between baseline cognitive functioning (BACS), negative symptoms (PANSS-negative scale), persecutory ideation (suspiciousness item from the PANSS), and FOCUS system use (ie, days used, number of times used per day). Spearman’s rank correlation coefficients were used to examine the association between reading level (WRAT-4 grade estimate) and FOCUS system use. Paired samples t tests were used to test for differences between pretrial and posttrial clinical outcomes (PANSS, BDI-2, BMQ, ISI).

Results

Feasibility

One participant dropped out of the study after losing 2 study smartphones in the first week. The remaining 32 participants used the system successfully and returned the smartphone intact at the end of the trial. One participant did not use the smartphone for anything other than the FOCUS intervention. All other participants used a range of smartphone functions during the month,

Medication Use          Mood              Sleep

including calling (96.9%), texting (34.4%), email (15.2%), and accessing the internet (62.5%). System use data for 2 participants were lost due to technical problems during the automated data transfer to the study server.

Therefore, we report on FOCUS system use data for 30 individuals.

On average, these participants used the FOCUS system on 86.5% of the days they had the smartphone (week 1 average: 6.7 days, week 4 average: 5.9 days). On days FOCUS was used, participants interacted with the system an average of 5.19 times (week 1 average: 6.4 times daily, week 4 average: 4.9 times daily). Participants initiated their interactions with FOCUS on 62.5% of the times it was used (ie, using on-demand interventions) and 37.5% of use was in response to prescheduled system prompts.

Baseline cognitive functioning (ie, BACS score), negative symptoms (PANSS-negative scale score), persecutory ideation (suspiciousness item from the PANSS), and reading level (WRAT-4 grade estimate) were not significantly associated with the percentage of days participants used FOCUS, or the number of times they used the system on those days (all Ps > .05). Overall, these results suggest the FOCUS smartphone intervention is feasible among people with schizophrenia.

Acceptability and Usability

Participant responses to the acceptability/usability measure are reported in table 1. Over 90% of participants thought the different components of the intervention worked well together, that content appeared on the screen clearly, and that people could learn to use FOCUS very quickly. They reported feeling very confident, comfortable, and satisfied using the intervention. Over 87% reported that it was easy to find the information they needed, that the intervention helped them manage their symptoms, and that they would recommend the system to a friend. A minority of participants reported some difficulties: approximately 12% found the intervention to be complicated, 18% thought they needed to learn more things before they could get started and found it awkward to use, and 6% thought they needed more technical support. Overall, these results suggest that the majority of participants found the FOCUS intervention acceptable and usable.

Preliminary Efficacy

Paired samples t tests indicated significant reductions in symptoms from pretrial to posttrial on the PANSS total (P = .002), PANSS positive (P < .001), PANSS general psychopathology (P < .001), and in depression on the BDI-2 (P = .003). Scores on the PANSS-negative subscale did not significantly change. There were also no significant changes in beliefs about medications (BMQ general and BMQ necessity-concern differential scores) or in sleep difficulties (ISI) (see table 2).

To examine whether there was an association between symptom change and the frequency with which participants used the intervention, we conducted Pearson correlations between changes in BDI-2 and PANSS scores, and the percentage of days participants used the system.

Table 2. Pre- and Posttrial Clinical Measures (N = 32)

*P < .01, **P < .001 (paired samples t tests).

We found a significant association between the change in participants’ BDI-2 scores and the percentage of days participants used FOCUS over the 1-month period (r = -.36, P < .05); the greater the reduction in depressive symptoms, the less often participants used the system. There was no significant association between PANSS scores and the percentage of days participants used the system.

Discussion

This study demonstrated that a smartphone intervention for illness management in individuals with schizophrenia is feasible and acceptable, and suggests the system may be clinically helpful. Participants reported high levels of satisfaction and an interest in continuing to use the smartphone system in the future. Acceptability of the smartphone intervention was high; participants used the system 86% of days they had the device. Additionally, individuals elected to use on-demand resources above and beyond the preprogrammed daily prompts; participant-initiated engagement accounted for over 60% of all intervention use. This finding is particularly important, given that participants were not given incentives to use the intervention during the trial and their access to other smartphone resources (ie, calling, texting, internet) was not dependent on their use of the FOCUS system. Participants’ use of the system was not hampered by their level of cognitive functioning, negative symptoms, persecutory ideation, or reading level.

Preliminary assessment of clinical efficacy suggested that the FOCUS intervention was helpful in reduction of positive symptoms of schizophrenia (PANSS-positive scale), general symptoms of psychopathology (PANSS general psychopathology scale), and depression (BDI-2) over the 1-month trial. There were no significant changes in sleep or beliefs about medications. Unlike the symptom-focused content, the smartphone system’s sleep interventions were designed to promote long-term lifestyle changes (ie, sleep hygiene) rather than in-the-moment coping strategies and might not provide immediate relief when deployed. For example, suggesting that an individual avoid naps during the day in order to facilitate a regular sleep/wake cycle may be less helpful in the moment when the person is struggling to fall asleep. It is also possible that positive sleep outcomes take longer than a month to emerge, and a longer trial would have produced stronger effects. At the beginning of the study most participants had stronger beliefs that medications were necessary for them to stay healthy than they had concerns about their use (BMQ necessity-concern differential), making it difficult to detect further increases in the importance of medication.

Overall, the clinical outcomes reported in this study are comparable or better than those produced by other psychosocial interventions^9,10 and required a fraction of the cost. Even after accounting for the price of smartphone devices (and their possible replacement), data plans, and technical support staff needed to deploy FOCUS, it is much less resource intensive than services provided by a mental health professional at a clinic setting. Moreover, unlike scheduled face-to-face services, a mobile intervention is transportable and can be used in any location.

While the FOCUS system was used in the study as an adjunct to in-person services, one could also envision a future where evidence-based mHealth apps such as FOCUS are downloaded directly onto smartphones and used by individuals with little or no access to any mental health care. As familiarity with mobile technology increases, so may the range of potentially therapeutic options for using them to promote coping with psychiatric illnesses. For people who are now growing up with mobile technologies in hand (ie, “digital natives”) using the full range of smartphone capabilities will be intuitive (eg, uploading photos, using tools to adapt or generate their own content, connecting to social media). As a result, the level of sophistication and potential impact of mHealth interventions (ie, tailored, personalized, adaptive) will increase.

This study has several limitations. First, there was no control group, so it is not possible to determine whether the clinical improvements were related to use of the FOCUS system. Future research will need to evaluate FOCUS in more rigorously controlled studies and to examine whether symptom improvements persist over time. Second, the system was deployed for a relatively short amount of time. While it is encouraging to see some rapid therapeutic gains, future research will need to examine whether people find smartphone mHealth systems engaging and helpful over extended periods, or whether these are most suitable for time-limited care (eg, upon discharge from inpatient care, during symptom exacerbations). Third, the sample size was relatively small, impacting generalizability. It is also possible that a larger sample would be sufficiently powered to detect small changes in clinical outcomes that were not significant in the current study (eg, sleep ratings). However, whether these small improvements are clinically meaningful, is questionable. Finally, the clinical rater was not blinded to the study objectives and the nature of the intervention. This might have impacted their ratings on posttrial measures that require some clinical interpretation (ie, PANSS).

Clinical researchers have begun developing a number of other novel technologies to improve the accessibility and quality of care available to people with schizophrenia.^42,43 Several of these approaches include web-based self-paced cognitive behavioral interventions for auditory hallucina-tions,⁴⁴ online peer support and social therapy for first-episode psychosis,⁴⁵ internet-based family intervention programs,^46,47 computerized “relational agents” designed to enhance medication adherence and physical activity,⁴⁸ and virtual reality paradigms for vocational rehabilita-tion^49,50 and treatment of persecutory ideation.⁵¹ Many of these approaches can be adapted to smartphone and other mobile platforms that would allow patients to use them wherever and whenever they need them the most.

Conclusion

To our knowledge, this is the first study to demonstrate the feasibility, acceptability, and efficacy of a smartphone intervention for schizophrenia. Incorporating a user-centered approach in the intervention development process was essential to generating a system that can address many of the unique obstacles that people with schizophrenia face when attempting to engage in mHealth treatment. The integration of several adapted evidence-based psychosocial intervention strategies into a single mobile platform appeared promising. But the FOCUS system does not merely serve as a delivery system for existing interventions. Rather, it introduces a novel approach to clinical care for schizophrenia (ie, real-time, real-place, on-demand, self-navigated), ie, only made possible through recent advancements in mobile hardware, software, and telecommunication infrastructure. As smartphone and other mobile technologies continue to develop, they will create new and exciting opportunities for innovative mHealth systems that will enable continuous assessment and treatment. Leveraging both active (ie, self-report of symptoms and functioning) and passive (ie, sensors that capture behavior and physiology) patient data will undoubtedly increase the potency of treatments for people with schizophrenia in the years ahead.

Funding

National Institute of Mental Health (R34MH100195, Principal Investigator: D.B.-Z.); Dartmouth SYNERGY Center for Clinical and Translational Science.

Acknowledgment

The authors have declared that there are no conflicts of interest in relation to the subject of this study.

References

• 1. Murray CJ, Lopez AD. The Global Burden of Disease. Geneva: World Health Organization; 1996.

• 2. Knapp M, Mangalore R, Simon J. The global costs of schizophrenia. Schizophr Bull. 2004;30:279-293.

• 3. Buckley PF, Miller BJ, Lehrer DS, Castle DJ. Psychiatric

comorbidities and schizophrenia. Schizophr   Bull.

2009;35:383-402.

• 4. Folsom DP, Hawthorne W, Lindamer L, et al. Prevalence and risk factors for homelessness and utilization of mental health services among 10,340 patients with serious mental illness in a large public mental health system. Am J Psychiatry. 2005;162:370-376.

• 5. Brekke JS, Prindle C, Bae SW, Long JD. Risks for individuals with schizophrenia who are living in the community. Psychiatr Serv. 2001;52:1358-1366.

• 6. Sun SX, Liu GG, Christensen DB, Fu AZ. Review and analysis of hospitalization costs associated with antipsychotic nonadherence in the treatment of schizophrenia in the United States. Curr Med Res Opin. 2007;23:2305-2312.

• 7. Ascher-Svanum H, Zhu B, Faries DE, et al. The cost of relapse and the predictors of relapse in the treatment of schizophrenia. BMC Psychiatry. 2010;10:2.

• 8. Hor K, Taylor M. Suicide and schizophrenia: a systematic review of rates and risk factors. J Psychopharmacol. 2010;24:81-90.

• 9. Dixon LB, Dickerson F, Bellack AS, et al.; Schizophrenia Patient Outcomes Research Team (PORT). The 2009 schizophrenia PORT psychosocial treatment recommendations and summary statements. Schizophr Bull. 2010;36:48-70.

• 10. Mueser KT, Deavers F, Penn DL, Cassisi JE. Psychosocial treatments for schizophrenia. Annu Rev Clin Psychol. 2013;9:465-497.

• 11. Drake RE, Bond GR, Essock SM. Implementing evidencebased practices for people with schizophrenia. Schizophr Bull. 2009;35:704-713.

• 12. Berry K, Haddock G. The implementation of the NICE guidelines for schizophrenia: barriers to the implementation of psychological interventions and recommendations for the future. Psychol Psychother. 2008;81:419-436.

• 13. Mojtabai R, Fochtmann L, Chang SW, Kotov R, Craig TJ, Bromet E. Unmet need for mental health care in schizophrenia: an overview of literature and new data from a first-admission study. Schizophr Bull. 2009;35:679-695.

• 14. Ben-Zeev D. Mobile technologies in the study, assessment, and treatment of schizophrenia. Schizophr Bull. 2012;38:384-385.

• 15. Luxton D, McCann R, Bush N, Mishkind M, Reger G. mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof Psychol Res Pr. 2011;42:505-512.

• 16. Proudfoot J. The future is in our hands: the role of mobile phones in the prevention and management of mental disorders. Aust N Z J Psychiatry. 2013;47:111-113.

• 17. International Telecommunication Union. The World in 2011: ICT Facts and Figures. 2011. Retrieved from http://www.itu. int/ITU-D/ict/facts/2011/material/ICTFacts Figures2011.pdf.

• 18. Smith A. Pew Internet and American Life Project: Americans and Their Cell Phones. Published 2011. Retrieved April 25, 2013 from http://pewinternet.org/Reports/2011/Cell-Phones.aspx

• 19. Eyrich-Garg KM. Mobile phone technology: a new paradigm for the prevention, treatment, and research of the non-shel-tered “street” homeless? J Urban Health. 2010;87:365-380.

• 20. Post LA, Vaca FE, Doran KM, et al. New media use by patients who are homeless: the potential of mHealth to build connectivity. J Med Internet Res. 2013;15:e195.

• 21. Ben-Zeev D, Davis KE, Kaiser S, Krzsos I, Drake RE. Mobile technologies among people with serious mental illness: opportunities for future services. Adm Policy Ment Health. 2013;40:340-343.

• 22. Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, Holden JL. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull. 2012;38:414-425.

• 23. Spaniel F, Hrdlicka J, Novak T, et al. Effectiveness of the information technology-aided program of relapse prevention in schizophrenia (ITAREPS): a randomized, controlled, double-blind study. JPsychiatr Pract. 2012;18:269-280.

• 24. Garety PA, Kuipers E, Fowler D, Freeman D, Bebbington PE. A cognitive model of the positive symptoms of psychosis. Psychol Med. 2001;31:189-195.

• 25. Freeman D, Garety PA, Kuipers E, Fowler D, Bebbington PE. A cognitive model of persecutory delusions. Br J Clin Psychol. 2002;41:331-347.

• 26. Liberman RP, Mueser KT, Wallace CJ, Jacobs HE, Eckman T, Massel HK. Training skills in the psychiatrically disabled: learning coping and competence. Schizophr Bull. 1986;12:631-647.

• 27. Zubin J, Spring B. Vulnerability-a new view of schizophrenia. J Abnorm Psychol. 1977;86:103-126.

• 28. Mueser KT, Meyer PS, Penn DL, Clancy R, Clancy DM, Salyers MP. The Illness Management and Recovery program: rationale, development, and preliminary findings. Schizophr Bull. 2006;32(suppl 1):S32-S43.

• 29. Rotondi AJ, Sinkule J, Haas GL, et al. Designing websites for persons with cognitive deficits: design and usability of a psy-choeducational intervention for persons with severe mental illness. Psychol Serv. 2007;4:202-224.

• 30. Ben-Zeev D, Kaiser SM, Brenner CJ, Begale M, Duffecy J, Mohr DC. Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatr Rehabil J. 2013;36:289-296.

• 31. Wilkinson GS, Robertson GJ. Wide Range Achievement Test—Fourth Edition. Lutz, FL: Psychological Assessment Resources; 2006.

• 32. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13:261-276.

• 33. Beck AT, Steer RA, Brown GK. Manual for Beck Depression Inventory II (BDI-II). San Antonio, TX: Psychology Corporation; 1996.

• 34. Morin CM, Belleville G, Belanger L, Ivers H. The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep. 2011;34:601-608.

• 35. Svarstad BL, Chewning BA, Sleath BL, Claesson C. The Brief Medication Questionnaire: a tool for screening patient adherence and barriers to adherence. Patient Educ Couns. 1999;37:113-124.

• 36. Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP, Coughenour L. The brief assessment of cognition in schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr Res. 2004;68:283-297.

• 37. First MB, Spitzer RL, Gibbon M, Williams JBW Structured Clinical Interview for DSMI-IV Axis I Disorders-Patient Edition (SCID-I/P), Version 2.0. New York, NY: Biometrics Research, New York State Psychiatric Institute; 1995.

• 38. Brooke J. SUS-A quick and dirty usability scale. Usability Evaluation in Industry. 1996:189-194.

• 39. Lewis JR. Psychometric evaluation of the poststudy system usability questionnaire: the PSSUQ. In: Proceedings of the Human Factor Society Annual Meeting. 1992;36:1259.

• 40. Venkatech V, Davis FD. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci. 2000;46:186-204.

• 41. Lund AM. Measuring usability with the USE questionnaire. Usability and User Experience. 2001;8:8.

• 42. Alvarez-Jimenez M, Gleeson JF, Bendall S, et al. Internetbased interventions for psychosis: a sneak-peek into the future. Psychiatr Clin North Am. 2012;35:735-747.

• 43. Ben-Zeev D, Drake RE, Corrigan PW, Rotondi AJ, Nilsen W, Depp C. Using contemporary technologies in the assessment and treatment of serious mental illness. Am J Psychiatr Rehabil. 2012;15:357-376.

• 44. Gottlieb JD, Romeo KH, Penn DL, Mueser KT, Chiko BP. Web-based cognitive-behavioral therapy for auditory hallucinations in persons with psychosis: a pilot study. Schizophr Res. 2013;145:82-87.

. Alvarez-Jimenez M, Bendall S, Lederman R, et al. On the HORYZON: moderated online social therapy for longterm recovery in first episode psychosis. Schizophr Res. 2013;143:143-149.

. Glynn SM, Randolph ET, Garrick T, Lui A. A proof of concept trial of an online psychoeducational program for relatives of both veterans and civilians living with schizophrenia. Psychiatr Rehabil J. 2010;33:278-287.

. Rotondi AJ, Anderson CM, Haas GL, et al. Web-based psy-choeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr Serv. 2010;61:1099-1105.

. Bickmore TW, Puskar K, Schlenk EA, Pfeifer LM, Sereika SM. Maintaining reality: relational agents for antipsychotic medication adherence. Interact Comput. 2010;22:276-288.

. Bell MD, Weinstein A. Simulated job interview skill training for people with psychiatric disability: feasibility and tolerability of virtual reality training. Schizophr Bull. 2011;37(suppl

• 2):S91-S97.

. Tsang MMY, Man DWK. A virtual reality-based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophr Res. 2013;144:51-62.

. Freeman D. Studying and treating schizophrenia using virtual reality: a new paradigm. Schizophr Bull. 2008;34:605-610.

Original Paper

Transdiagnostic Mobile Health: Smartphone Intervention Reduces Depressive Symptoms in People With Mood and Psychotic Disorders

Dror Ben-Zeev¹, PhD; Benjamin Buck^1,2,3, PhD; Phuonguyen Vu Chu¹, BA; Lisa Razzano^4,5, CPRP, PhD; Nicole Pashka⁵, MS, CRC, CPRP, LCPC; Kevin A Hallgren¹, PhD

¹Behavioral Research In Technology and Engineering Center, Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, United States

²Health Services Research and Development, VA Puget Sound Healthcare System, Seattle, WA, United States ³Department of Health Services, School of Public Health, University of Washington, Seattle, WA, United States ⁴Center on Mental Health Services Research and Policy, Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, United States ⁵Thresholds Inc, Chicago, IL, United States

Corresponding Author:

Dror Ben-Zeev, PhD

Behavioral Research In Technology and Engineering Center

Department of Psychiatry and Behavioral Sciences

University of Washington

1959 NE Pacific Street

Seattle, WA, 98195

United States

Phone: 1 206 685 9655

Email: dbenzeev@uw.edu

Abstract

Background: Depression is the most prevalent mental health problem. The need for effective treatments for depression far outstrips the availability of trained mental health professionals. Smartphones and other widely available technologies are increasingly being leveraged to deliver treatments for depression. Whether there are patient characteristics that affect the potency of smartphone interventions for depression is not well understood.

Objective: This study aimed to evaluate whether patient characteristics including clinical diagnosis, depression severity, psychosis status, and current use of antidepressant medications impact the effects of an evidence-based smartphone intervention on depressive symptoms.

Methods: Data were collected as part of a 2-arm randomized controlled trial comparing a multimodal smartphone intervention called FOCUS with a clinic-based intervention. Here, we report on 82 participants assigned to 12 weeks of FOCUS treatment. We conducted assessments of depressive symptoms using the Beck Depression Inventory-second edition (BDI-II) at baseline, postintervention (3 months), and follow-up (6 months). We tested for differences in the amount of improvement in BDI-II scores from baseline to posttreatment and 6-month follow-up between each of the following patient subgroups using 2 (group) x 2 (time) mixed effects models: diagnosis (ie, schizophrenia spectrum disorder vs bipolar disorder vs major depressive disorder), depression severity (ie, minimal-mild vs moderate-severe depression), psychosis status (ie, presence vs absence of psychotic symptoms), and antidepressant use (ie, taking antidepressants vs not taking antidepressants).

Results: The majority of participants were male (60%, 49/82), African American (65%, 53/82), and middle-aged (mean age 49 years), with a high school education or lower (62%, 51/82). There were no differences in patient demographics across the variables that were used to stratify the analyses. There was a significant group x time interaction for baseline depression severity (F_b768=5.26, P=.02 [posttreatment] and Fj ₇₇₄=6.56, P=.01 [6-month follow-up]). Participants with moderate or severe depression had significant improvements (t₄₂=3.20, P=.003 [posttreatment] and /₄₂=4.20, P<.001 [6-month follow-up]), but participants with minimal or mild depression did not (t₃₁=0.20, P=.84 [posttreatment] and /₃₀=0.43, P=.67 [6-month follow-up]). There were no significant group x time interactions for diagnosis, psychosis status, or antidepressant medication use. Participants with minimal or mild

http://mental.jmir.org/2019/4/e13202/

XSL-FO

depression had negligible nonsignificant improvements (<1 point on the BDI-II). Reduction in depression in all other groups was larger (range 1.7-6.5 points on the BDI-II).

Conclusions: Our results suggest that FOCUS can be deployed to treat moderate to severe depressive symptoms among people with schizophrenia spectrum disorders, bipolar disorder, and major depressive disorder, in concert with antidepressant medications or without them, in both people with and without active psychotic symptoms. The study results are consistent with research on transdiagnostic models in psychotherapy and extend our knowledge about the potential of transdiagnostic mobile health.

Trial Registration: ClinicalTrials.gov NCT02421965; http://clinicaltrials.gov/ct2/show/NCT02421965 (Archived by WebCite at http://www.webcitation.org/76pyDlvAS)

(JMIR Ment Health 2019;6(4):e13202) doi:10.2196/13202

KEYWORDS

mHealth; schizophrenia; bipolar disorder; depression; illness management; symptoms; transdiagnostic

Introduction

Background

Depression is a universal experience. Symptoms of depression are the most common mental health concern reported across nationalities, ethnicities, and age groups worldwide [1,2]. Depression is prevalent in all clinical settings, either as the primary issue that brings people to seek mental health care [3,4] or as a significant comorbid concern that emerges in those contending with medical illnesses [5-8], physical disabilities [9,10], relationship problems [11], work and educational difficulties [12], or substance use problems [13]. Depression is a source of enormous financial and societal burden. It is the second leading cause of years lived with disability worldwide [14], with overall medical and job loss costs estimated at US $210.5 billion annually [15]. It is also implicated in half to two-thirds of all completed suicides [16]. The need for effective treatments for depression far outstrips the availability of resources that can be delivered by trained mental health professionals. This tension has led to the emergence of new models of care that are no longer dependent on the availability of highly trained mental health specialists, including integration of depression treatments into the offerings of primary care settings [17-19], training of paraprofessionals and family members to provide support [20], and using new technology to expand the breadth and reach of depression management resources [21,22].

It is only fitting that in recent years the most common mental health problem is starting to be addressed with the aid of one of the most widely used technologies on the planet: the mobile phone [23,24]. Smartphones—contemporary mobile phones with multimedia players, internet connectivity, and the ability to host apps—are increasingly being leveraged to deliver treatments for depression. A recent meta-analysis of 18 randomized controlled trials (RCTs) of smartphone-based mental health interventions for depressive symptoms found that these treatments had positive effects in comparison with both active interventions and inactive control conditions. Therapeutic effects were found for those with self-reported mild to moderate depression but were not seen among those with diagnoses of major mood disorders [25]. The authors of the meta-analysis outlined that these findings may be linked to the small and underpowered subsample sizes used in the studies involving those with clinical diagnoses and emphasized the need for further research to deepen our understanding of which populations stand to benefit the most from smartphone interventions for depressive symptoms. Digital treatments are very novel in mental health care, and there is still uncertainty about whether these approaches are appropriate for all patients [26,27]. Research evaluating the effects of mobile health (mHealth) on people with both mild and more severe symptomatology can address this gap in our knowledge.

Objectives

Given that depressive symptoms commonly emerge in many forms of psychopathology [28], in this study, we examined the effects of an evidence-based smartphone intervention called FOCUS on depressive symptoms transdiagnostically among people with mood and psychotic disorders. We evaluated whether several patient clinical characteristics (ie, diagnosis, depression symptom severity, psychosis symptoms, and antidepressant medication use) impact the effects of the smartphone intervention on depressive symptoms in an RCT.

Methods

Study Descriptions

We conducted an assessor-blind, 2-arm, RCT between 2015 and 2017. The project was conducted in partnership with a large Chicago-based mental health agency that provides services to a range of people with psychiatric conditions. The study was approved by the Institutional Review Boards of the University of Washington and Dartmouth College and monitored continuously by an independent safety monitoring board. All study participants completed informed consent. Individuals were randomized (1:1 ratio) into 1 of the 2 treatment arms: an mHealth intervention delivered via smartphone (FOCUS) or a clinic-based group intervention (Wellness Recovery Action Plan). Interventions were deployed for a period of 12 weeks. We conducted assessments at baseline, postintervention (3 months), and follow-up (6 months). Participants were compensated $30 per assessment. The study was registered in ClinicalTrials.gov, and the main RCT comparison outcomes were reported in an earlier publication [29]. Here, we conducted a secondary analysis focusing specifically on patients who received the FOCUS smartphone intervention, examining whether several patient clinical characteristics preintervention affected the impact of the intervention on their depressive symptoms.

Smartphone Intervention

FOCUS is a multimodal, smartphone-delivered intervention that was originally designed to support the recovery of people with schizophrenia [30] but has since been deployed among multiple diagnostic groups [29,31,32]. The FOCUS intervention comprises a FOCUS app that is used independently by patients, a Web-based clinician dashboard that summarizes participants’ responses to self-assessments and their use of various FOCUS resources, and an mHealth support specialist who helps participants make meaningful use of the FOCUS intervention and provides technical troubleshooting assistance via brief weekly phone calls [32]. FOCUS treatment content targets 5 broad domains: mood (ie, depression and anxiety), voices (ie, auditory hallucinations), sleep problems, medication use, and social functioning. Content takes the form of either brief video, audio, or sequences of digital screens with written material coupled with visual displays. The FOCUS app includes preprogrammed daily prompts (questions that take over the home screen), followed by tailored intervention content. Participants who identify as having significant difficulties with depression at baseline may be assigned mood focused prompts, but all content is accessible on demand to all users 24/7 without restriction.

Participants

A total of 82 participants who were assigned to the FOCUS smartphone arm in the RCT are included in this report; posttreatment and 6-month follow-up data were available for 75 (91.5%) and 74 (90.2%) participants, respectively. Participants were identified by research staff and clinical teams at 3 agency sites. Study inclusion criteria included the following: chart diagnosis of schizophrenia, schizoaffective disorder, bipolar disorder, or major depressive disorder; aged 18 years or older; and a rating of “3” or lower on 1 of the 3 items comprising the Domination by Symptoms factor from the Recovery Assessment Scale [33]. Exclusion criteria included hearing, vision, or motor impairment (ie, that could affect the operation of a smartphone); less than 5th-grade English reading ability (per the Wide Range Achievement Test-4) [34]; and exposure to study interventions in the past 3 years. Participants continued to be eligible for all other clinical services including crisis intervention, assertive community treatment, supported employment, psychiatric evaluation, medication monitoring, psychosocial rehabilitation, and case management. Services were delivered in-person in the community or at 1 of the agency’s multiple locations.

Measures

The primary outcome (depression symptoms) was measured with the Beck Depression Inventory-second edition (BDI-II) [35]. The BDI-II is a self-report questionnaire with 21 items rated on a 4-point scale that can be summed for a continuous total depression severity score ranging from 0 to 63; scores can also be categorized to characterize symptom severity (0-13=minimal, 14-19=mild, 20-28=moderate, and 29-63=severe). For subgroup analyses, participants were categorized based on whether they had minimal to mild depression versus moderate to severe depression at baseline. Psychotic symptoms were assessed with the Psychotic Symptom Rating Scales (PSYRATS) [36], a semistructured interview instrument that assesses the severity of auditory hallucinations (eg, frequency, duration, loudness, and distress) and delusions (eg, preoccupation, conviction, and disruption). The PSYRATS comprises 17 items, each rated on a 4-point scale and summed for a total psychotic symptoms score. Given the distribution of psychotic symptoms at baseline in our sample (63.8% endorsing none), we dichotomized our sample based on whether individuals had any psychotic symptoms (vs none) at baseline. Antidepressant medication use was recorded by study assessors during baseline interviews and follow-up calls where participants were asked to read their medication labels to study staff. We dichotomized our sample based on whether study participants were actively taking any antidepressant medications (vs not) before commencing study interventions. Participants’ diagnosis was recorded from the electronic health records. Diagnoses are determined by licensed clinical social workers or licensed clinical professional counselors who interview clients about their mental health challenges and history, examine any prior medical records, and consult with the agency’s medical director who is a board-certified psychiatrist.

Data Analytic Plan

We conducted a series of 2 (group) x 2 (time) mixed effects models to evaluate whether there were differences in the amount of clinical improvement in BDI-II scores between each of the following groups: (1) Diagnosis, schizophrenia spectrum disorder versus bipolar disorder versus major depressive disorder; (2) Depression severity, minimal-mild versus moderate-severe depression; (3) Psychosis symptoms, presence versus absence of psychotic symptoms, and (4) Antidepressant medications, taking antidepressants versus not taking antidepressants. Any significant group x time interactions would indicate that the amount of clinical improvement was moderated by the baseline group variable; interactions that were significant were followed by paired sample t tests to evaluate the significance of the amount of changes in BDI-II scores within each specific group.

Results

Demographics and Study Variables

Descriptive statistics are presented in Table 1. The majority of participants were male and African American; the mean age was 49 years. Most participants had a high school education or lower (62%, 51/82) and had used a smartphone before entering the study (73%, 60/82). The 3 diagnostic categories specified as inclusion criteria (schizophrenia, schizoaffective disorder, bipolar disorder, and major depressive disorder) were well represented within the sample. There were no differences in patient demographics across the baseline measures that were used to stratify our analyses (diagnosis, depression severity, psychosis symptoms, and antidepressant medications).

Table 1. Descriptive statistics at baseline (N=82).

http://mental.jmir.org/2019/4/e13202/

XSL-FO

Discussion

Principal Findings

The results of this study deepen our understanding of patient clinical characteristics that may impact the effectiveness of FOCUS on their level of depression, independent of their assigned diagnostic label. We found that (1) FOCUS produced significant and sustained (at 6-month follow-up) reduction in depression among people who had moderate to severe depressive symptoms, effects that were not seen among people with minimal to mild depressive symptoms; (2) FOCUS produced significant and sustained reductions in depression among people with schizophrenia/schizoaffective disorder, bipolar disorder, and major depressive disorder; (3) FOCUS produced significant and sustained reductions in depression among people with psychotic symptoms and among people without any indication of psychosis; and (4) FOCUS produced significant and sustained reduction in depression in both people who were taking antidepressant medications and people who were not.

Our findings suggest that FOCUS might be a useful intervention to address moderate to severe depressive symptoms among individuals with an array of mental illnesses. Depressive symptoms are common among people experiencing psychosis [37], are linked with poorer outcomes [38,39], and often persist or recur even with antidepressant treatment [40]. Our findings go against current skepticism about the viability of computerized interventions for people with psychosis [26] as FOCUS produced significant positive effects in both individuals with and without active psychotic symptoms and in both people with and without a schizophrenia spectrum diagnosis.

In the context of precision medicine in mental health care [41] and the growing interest in customization of treatment for well-defined populations, this study can inform practical clinical decision making. Our results suggest that FOCUS can be deployed to effectively treat depression transdiagnostically among people with moderate to severe depressive symptoms, in concert with antidepressant medications or without them, in both people with and without co-occurring psychotic symptoms.